Using MATA in nonlinear-least-square nl command.

[kilasuelika sue]

Following codes use nl to fit a quadratic regression and using mata to do calculation:

clear
set obs 1000
gen x=2*(runiform()-0.5)
gen y=0.6*x^2+0.3+rnormal()
gen y1=.

mata: mata clear
mata:
    function squad(string scalar xvar, string scalar yvar,  string scalar mat, string scalar touse){
        
        coeff=st_matrix(mat)
        a=coeff[1,1]
        b=coeff[1,2]
        
        st_view(x=., ., xvar, touse)    
        st_view(y1=., ., "y1", touse)  //use this variable to track tempvar
        
        _y=a :* x :^2 :+b
        y1[.]=_y
        
        st_view(y=., ., yvar, touse)  
        y[.]=_y //directly modify temporary var.
    }
    
end


cap program drop nlquadrege

program define nlquadrege
    version 17
    syntax varlist(min=2 max=2)   [aw fw iw] if, at(name)  
    
    local y : word 1 of `varlist'
    local x : word 2 of `varlist'
    
    marksample touse
    
    mata : squad("`x'","`y'", "`at'", "`touse'")
    
    replace `y'=y1  //If remove this line, there will be error.
end

nl quadrege @ y x , parameters(a b) initial(a 1 b 1)  log trace

In mata, I directly set value of `y'. But the problem is that, if I remove the line replace `y'=y1 (this line set values again), the nl command wil fail. I'm curious about why I can't directly change values in mata.

[Brian Poi]

Here is a working implementation of your code:

clear
set obs 1000
gen x=2*(runiform()-0.5)
gen y=0.6*x^2+0.3+rnormal()
gen y1=.

mata: mata clear
mata:
    function squad(string scalar xvar, string scalar yvar,  string scalar mat,
                   string scalar touse){
        
        coeff=st_matrix(mat)
        a=coeff[1,1]
        b=coeff[1,2]
        
        st_view(x=., ., xvar, touse)    
        st_view(y=., ., yvar, touse)  

        y[.] = a :* x :^2 :+b
    }
    
end


cap program drop nlquadrege

program define nlquadrege
    version 17
    syntax varlist(min=2 max=2)   [aw fw iw] if, at(name)  

    tempvar touse
    qui gen `touse' = 0
    qui replace `touse' = 1 `if'
        
    local y : word 1 of `varlist'
    local x : word 2 of `varlist'

    mata : squad("`x'","`y'", "`at'", "`touse'")
    
end

nl quadrege @ y x , parameters(a b) initial(a 1 b 1)  log trace

Notice that I did not use -marksample- to create the touse variable but instead created it manually. For reasons that are utterly baffling to me, if I use -marksample-, the touse variable contains all zeros; and the -nl- command fails because the LHS variable does not get filled in. I'm entirely befuddled as to why -marksample- does not work here.

[kilasuelika sue]

Here is a working implementation of your code:

clear
set obs 1000
gen x=2*(runiform()-0.5)
gen y=0.6*x^2+0.3+rnormal()
gen y1=.

mata: mata clear
mata:
function squad(string scalar xvar, string scalar yvar, string scalar mat,
string scalar touse){

coeff=st_matrix(mat)
a=coeff[1,1]
b=coeff[1,2]

st_view(x=., ., xvar, touse)
st_view(y=., ., yvar, touse)

y[.] = a :* x :^2 :+b
}

end


cap program drop nlquadrege

program define nlquadrege
version 17
syntax varlist(min=2 max=2) [aw fw iw] if, at(name)

tempvar touse
qui gen `touse' = 0
qui replace `touse' = 1 `if'

local y : word 1 of `varlist'
local x : word 2 of `varlist'

mata : squad("`x'","`y'", "`at'", "`touse'")

end

nl quadrege @ y x , parameters(a b) initial(a 1 b 1) log trace

Notice that I did not use -marksample- to create the touse variable but instead created it manually. For reasons that are utterly baffling to me, if I use -marksample-, the touse variable contains all zeros; and the -nl- command fails because the LHS variable does not get filled in. I'm entirely befuddled as to why -marksample- does not work here.

Thank you, it works perfectly.

[Joseph Coveney]

Notice that I did not use -marksample- to create the touse variable but instead created it manually. For reasons that are utterly baffling to me, if I use -marksample-, the touse variable contains all zeros; and the -nl- command fails because the LHS variable does not get filled in. I'm entirely befuddled as to why -marksample- does not work here.

I think that -nl- is feeding the evaluator a temporary variable that's all-missing-valued, and that's why the touse variable is all-zero. If you use the -novarlist- option of -marksample-, then you'll avoid that and the code will work. See the code below, which runs without error. (I've cleaned things up a little bit.)

version 17.0

clear *

// seedem
set seed 1415659066

program define nlquadrege
    version 17.0
    syntax varlist(min=2 max=2) if, at(name)  

    marksample touse, novarlist

    gettoken y x : varlist

    mata: squad()
    
end

mata:
mata set matastrict on

void function squad() {

    real rowvector Coef
    Coef = st_matrix(st_local("at"))

    real matrix Data
    pragma unset Data
    st_view(Data, ., (st_local("x"), st_local("y")), st_local("touse"))
    Data[., 2] = Coef[1] :* Data[., 1] :^2 :+ Coef[2]
}
    
end

*
* Begin here
*
quietly set obs 1000

generate double x = runiform(-1, 1)
generate double y = 0.6 * x^2 + 0.3 + rnormal()

nl quadrege @ y x, parameters(a b) initial(a 1 b 1)

// cf.
nl (y = {a}*x^2 + {b})

exit

In case there are missing values in the predictor list, I assume that -nl- will mark them out in the -if- condition and pass that through to the evaluator program. But if not (I can't imagine why -nl- wouldn't), then you could do it inside the evaluator program with something like

markout `touse' `x'

after the -marksample- line.