conformability error

[DL Shin]

Hello, I'm creating an index based on corporate financial, stock, and emissions data. I'm currently using Stata version 14.1.

I've finished processing the data, and I'm trying to calculate company-specific weights by running optimizations monthly. Below is the code I wrote for this process.

I'm running mata within a loop, and I'm getting a conformability error (r3200). I thought the loop was the problem, but I keep getting the same error when I run the code for specific months and years.
When I specify w as a colvector (i.e., void obj(todo, real colvector w, f, g, H), I get the following error:
obj(): 3203 <tmp>[1,77] found where colvector required

I can't figure out where the problem is. Please help.

capture erase "results_intermediate.dta"

capture erase "barra_pab_final_prevyear_penalty.dta"

save "results_intermediate.dta", replace emptyok

 

levelsof mdate, local(months)

local WACI_prev = .

local end_mata end

 


foreach t of local months {

    di as txt "===== 최적화 시작: `t' ====="

    preserve

 

    keep if mdate == `t' & exclusion == 0

    keep company_id mdate intensity kospi_intensity ret_kospi ret_company w_b manufacturing style*

 

    mata:

                        WB = st_data(., "w_b")

                        R = st_data(., "ret_company")

                        CI = st_data(., "intensity")

                        CIK = st_data(., "kospi_intensity")

                        MAN = st_data(., "manufacturing")

                        E = st_data(., ("style1","style2","style3","style4","style5","style6","style7","style8","style9","style10","style11"))

                        N = rows(WB)

                        Rb_vec = st_data(., "ret_kospi")

                        WACI_prev = st_numscalar("WACI_prev")

                        Rb=Rb_vec[1]

 

 

        void obj(todo, real colvector w, f, g, H)

        {

                        f = (R'*w - Rb)^2

            pen = 0

 

            // (1)

            man_bench = MAN' * WB

            man_weight = MAN' * w

            if (man_weight < man_bench) pen = pen + 1000 * (man_bench - man_weight)

 

            // (2)

                        waci_bench = WB' * CIK

            carbon_weight = CI' * w

            if (carbon_weight > 0.5 * waci_bench) pen = pen + 1000 * (carbon_weight - 0.5 * waci_bench)

 

            // (3)

            if (WACI_prev != .) {

                if (carbon_weight > 0.93 * WACI_prev) pen = pen + 1000*(carbon_weight - 0.93*WACI_prev)

            }

                        

            f = f + pen

            if (todo >= 1) g = 2*(R'*w - Rb)*R

            if (todo >= 2) H = 2*(R*R')

        }

 

        M = optimize_init()

        optimize_init_evaluator(M, &obj())

        optimize_init_which(M, "min")

        optimize_init_params(M, WB') // 행벡터 입력

        optimize_init_conv_maxiter(M, 1000)

 

                

        Aeq = J(1, N, 1)

        beq = (1)

        Ftarget = E' * WB              //(N x 11)' * (N x 1) = (11 x 1)

        Aeq = Aeq \ E'                 //(12 x 77) (\는 행결합, 즉 세로 머지)

        beq = beq \ Ftarget            //(12 x 1)

        Cc = Aeq, beq                  //(12 x 78)

        optimize_init_constraints(M, Cc)

 

        w_opt = optimize(M)

        st_store(., "w_opt", w_opt)

    

        WACI_now = (CI'*w_opt)

        st_numscalar("WACI_now", WACI_now)

        

        `end_mata'

    

    scalar WACI_prev = r(WACI_now)

 

    gen mdate = `t'

    append using "results_intermediate.dta"

    save "results_intermediate.dta", replace

    restore

        }



use "results_intermediate.dta", clear

order mdate company_id w_opt

save "barra_pab_final_prevyear_penalty.dta", replace

export delimited using "barra_pab_final_prevyear_penalty.csv", replace

[Joseph Coveney]

When I specify w as a colvector (i.e., void obj(todo, real colvector w, f, g, H), I get the following error:
obj(): 3203 <tmp>[1,77] found where colvector required

I can't figure out where the problem is. Please help.

According to the documentation for optimize(), the parameter vector that optimize() feeds to the evaluator function is a row vector, and so the evaluator function's argument must be coded so as to accept a row vector.

You can transpose it as needed inside the evaluator function.

[DL Shin]

Thank you for your comment, Joseph Coveney.

I understood your comment to mean that when defining the obj() function, w should be recognized as a rowvector. (For example, fixing f=(R'*w - Rb)^2 to f(R'*w' - Rb)^2.)
The code still fails with a conformability error even with such modification.

[Joseph Coveney]

I understood your comment to mean . . . fixing f=(R'*w - Rb)^2 to f(R'*w' - Rb)^2.)

No, that's not what I meant.

I meant the following:

void obj(. . . , real rowvector w, . . . )

Inside the function you would need to transpose it only once, at the top before its first use:

w = w'

and then the rest of your code can remain the same.

[Joseph Coveney]

Inside the function you would need to transpose it only once, at the top before its first use. . .

It looks as if the parameter vector is used later by optimize() and because functions' arguments are passed by reference in Mata, you need to maintain the parameter vector that is fed to the evaluator function as a row vector.

So, what I recommended above needs to be slightly modified:

void obj(todo, real rowvector wa, f, g, H)
        {
                        w = wa'
                        // and then you can proceed with the remaining code unchanged
                        f = (R'*w - Rb)^2

By the way, your function computes the gradient vector and Hessian matrix, but you didn't set the evaluator type for optimize() to take advantage of that. If you want optimize() to utilize those, then you'll need to add a line instructing it to

M = optimize_init()
optimize_init_evaluator(M, &obj())
optimize_init_evaluatortype(M, "d2")
optimize_init_which(M, "min")

or did you deliberately omit that line for diagnostic purposes?

[DL Shin]

Thank you Joseph Coveney

Through several experiments, I confirmed that the conformability error occurs when calculating f=w*R. I specified w as real rowvector and R as a colvector, so why is still there a problem with the calculation? (when I run mata: mata des, R appears as a real colvector.)

// Added evaluatortype. Thanks

[Joseph Coveney]

I confirmed that the conformability error occurs when calculating f=w*R.

That conformability error is a different one from the one whose error message you showed above in #1.This type of nonconformability will give an error message of

ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ*:ÿÿ3200ÿÿconformabilityÿerror
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ<istmt>:ÿÿÿÿÿ-ÿÿfunctionÿreturnedÿerror
r(3200);

while the one in your original post refers to a "<tmp>[1,77] found where colvector required" with an error number of 3203, which is what you get when you feed a row vector as an argument to a function that specifies a column vector for its argument.

I specified w as real rowvector and R as a colvector, so why is still there a problem with the calculation?

I'm guessing that it's because the dimensions differ. I'm not sure what you're trying to do, but according to the error message in your original post, w is 77 elements long, and R is a column vector made from one of the variables in your dataset, perhaps dozens or hundreds of observations.