Help with error in FMM - Stata 15

[Alejandro Gonzalez]

Dear All,

When using the new FMM package for stata 15, I get a strange error which I'm not able to identify. I'll copy and paste both my model and the output.

. fmm 2, lcprob(lnYL esc edad edad2 sexo i.pais i.ano): regress lnw esc_bas esc_bas_m esc_ter esc_ter_m edad edad2 sexo i.pais i.ano if
> Obs_Isic>24 & lnYL!=. & pais!=3 & area_ee==1 & clase==2 & inrange(edad,25,60) // As you can see, I have both discrete and continous variables in both the selection and outcome equations.

Fitting class model:

Iteration 0: (class) log likelihood = -82460.947
Iteration 1: (class) log likelihood = -14760.93
Iteration 2: (class) log likelihood = -13874.606
Iteration 3: (class) log likelihood = -12556.455
Iteration 4: (class) log likelihood = -12504.097
Iteration 5: (class) log likelihood = -12504.067
Iteration 6: (class) log likelihood = -12504.067

Fitting outcome model:

Iteration 0: (outcome) log likelihood = -104871.29
Iteration 1: (outcome) log likelihood = -104871.29

Refining starting values:

Iteration 0: (EM) log likelihood = -118325.46
Iteration 1: (EM) log likelihood = -118459.52
Iteration 2: (EM) log likelihood = -118479.1
Iteration 3: (EM) log likelihood = -118507.43
Iteration 4: (EM) log likelihood = -118560.33
Iteration 5: (EM) log likelihood = -118634.26
Iteration 6: (EM) log likelihood = -118723.53
Iteration 7: (EM) log likelihood = -118823.72
Iteration 8: (EM) log likelihood = -118931.97
Iteration 9: (EM) log likelihood = -119046.61
Iteration 10: (EM) log likelihood = -119166.73
Iteration 11: (EM) log likelihood = -119291.91
Iteration 12: (EM) log likelihood = -119422.08
Iteration 13: (EM) log likelihood = -119557.33
Iteration 14: (EM) log likelihood = -119697.91
Iteration 15: (EM) log likelihood = -119844.19
Iteration 16: (EM) log likelihood = -119996.6
Iteration 17: (EM) log likelihood = -120155.66
Iteration 18: (EM) log likelihood = -120321.97
Iteration 19: (EM) log likelihood = -120496.24
Iteration 20: (EM) log likelihood = -120679.27
Note: EM algorithm reached maximum iterations.

Fitting full model:

_gsem_eval_mix__wrk(): 3900 unable to allocate real <tmp>[118966,754]
_gsem_eval_mix(): - function returned error
mopt__calluser_v(): - function returned error
opt__eval_nr_v2(): - function returned error
opt__eval(): - function returned error
opt__looputil_iter0_common(): - function returned error
opt__looputil_iter0_nr(): - function returned error
opt__loop_nr(): - function returned error
_moptimize(): - function returned error
Mopt_maxmin(): - function returned error
<istmt>: - function returned error


I don't know what to do or what I'm doing wrong. Running the same model without the lcprob(z1 z2); that is, without the logit equation, the model runs fine.
I run the same model using the user-made fmm command in Stata 14 and had no problems whatsoever fitting it.

Regards,
Alejandro

[Weiwen Ng]

OK, googling the phrase "3900 unable to allocate real" leads me to a bunch of threads on the forum, and they seem to indicate some issue with you running out of memory, e.g. mata requested some sort of matrix that was simply too big for Stata to handle.

https://www.statalist.org/forums/for...mp-22901-24090
https://www.statalist.org/forums/for...with-xtoprobit
https://www.statalist.org/forums/for...message-r-3900
https://www.statalist.org/forums/for...ct-var-remodes

If this is in fact the issue, one thing to check first is if you specified that any continuous variable be treated as a factor variable (which will generate one dummy for each unique value of the variable). If that doesn't help you can try the -dnumerical- option, which, in the words of Stata's own Jeff Pitblado,

... causes gsem to compute the gradient vector
and Hessian matrix numerically instead of analytically. The result of this is
less memory required but slower model fit.

Honestly, I have no clue what this means. But, I can give you code! At least, I can give you code that I believe to be correct. The -fmm- prefix doesn't appear to support the dnumerical option, but you can use -gsem-:

gsem (C <- lnYL esc c.edad##c.edad sexo i.pais i.ano) ///
(lnw <- esc_bas esc_bas_m esc_ter esc_ter_m edad edad2 sexo i.pais i.ano) ///
if Obs_Isic>24 & lnYL!=. & pais!=3 & area_ee==1 & clase==2 & inrange(edad,25,60), ///
lclass(C 2) dnumerical nocapslatent latent(C)

The first line is your lcprob statement in the fmm prefix (also note, I typically encourage people to use factor variable syntax instead of manually generating interaction terms - may not matter here). The second line was your regress command.

Do note that Stata will usually do listwise deletion of observations with any missing variables in your regression. I believe that in -gsem- or in -fmm-, you won't need to manually specify that your dependent variable can't be missing.

[Alejandro Gonzalez]

Thanks!

Actually, had a very big database (3.500.000 observations), but was only using a subset of them during the estimation (180.000). Thus, what I did was to drop all of the observations that I wasn't using, increasing the memory available for stata, and I had no further problems.

Cheers!
Alejandro.

[Weiwen Ng]

I am glad this worked out!

[Niels Henrik Bruun]

A late comment since I had a similar problem with "EM algorithm reached maximum iterations."
In help you will find an option emopts. There you can change iterate from the default of maxiter (20) to eg 50.

[Niels Henrik Bruun]

It turns out that I didn't read original post properly. Sorry.