The Number of Iterations for Convergence in ml

[Shuye Yu]

I am writing a programme for the ml command. I found that even if a subtle difference in my codes can make a huge difference in the number of iterations for convergence. I want to know why this is the case.

Take a simple example. I wrote the codes for OLS in the following three ways:

clear all
set obs 3000
gen  i=_n

gen u=rnormal(0,0.5)
forval i=1 (1) 3{
    gen x`i'=rnormal(`i'-1,`i')
}
gen y=2*x1+4*x2-0.5*x3+u

capture program drop myols
program define myols
    args lnf xb sigma
    tempvar sp1
   quietly{
    gen `sp1'=normalden($ML_y,`xb',`sigma')
    replace `lnf'=ln(`sp1')
   }
end

ml model lf myols (y: y = x1 x2 x3) (sigma:)
ml maximize

capture program drop myols2
program define myols2
    args lnf xb lnsigma
    tempvar sp1
   quietly{
    gen `sp1'=normalden($ML_y,`xb',exp(`lnsigma'))
    replace `lnf'=ln(`sp1')
   }
end

ml model lf myols2 (y: y = x1 x2 x3) (lnsigma:)
ml maximize

capture program drop myols3
program myols3
  args lnf xb lnsigma
  quietly replace `lnf' = ln(normalden($ML_y, `xb',exp(`lnsigma')))
end


ml model lf myols3 (y: y = x1 x2 x3) (lnsigma:)
ml maximize

These three programming methods should be equivalent (the only exception is the first programme reports sigma and the other two report ln(sigma)). However, the first two programmes take 100+ iterations to converge (or even not converge in some cases), but the last one can converge very quickly. Can anyone explain to me why these three programmes have so many different results? Thanks!

[FernandoRios]

Hi Shuye
There are a couple of reasons for that.
1. The most important is data type storage. Whenever you use -ml-, if you create any variable within the program, you should always create it in double format. The default is to create a variable as float, which is not precise enough for the internal calculations and derivatives.
2. The first program is trying to obtain an estiamate for sigma directly. This is a problem because -ml- starts with the assumption that a reasonable guess for all parameters is a vector of zeros, which is not a good guess for the variance. This is why the first iteration shows "-<inf> couldnt not be evaluated."

HTH

[Shuye Yu]

Hi Shuye
There are a couple of reasons for that.
1. The most important is data type storage. Whenever you use -ml-, if you create any variable within the program, you should always create it in double format. The default is to create a variable as float, which is not precise enough for the internal calculations and derivatives.
2. The first program is trying to obtain an estiamate for sigma directly. This is a problem because -ml- starts with the assumption that a reasonable guess for all parameters is a vector of zeros, which is not a good guess for the variance. This is why the first iteration shows "-<inf> couldnt not be evaluated."

HTH

Thanks! It makes huge progress following your suggestions.