Error: "unable to allocate real" when trying to fit a mixed effects logistic regression model with nested random effects

[Marta Checchi]

Hi,

I'm trying to fit a mixed effects logistic regression model with nested random effects (patient nested within clinic), with the following code:

melogit outcome_var exposure1_var i.exposure2_var exposure3_var || clinic: || patient_id: , or


My dataset is almost 4 million records, and the model takes forever to run, but ultimately it did not run (after a day to get through 35 iterations) I got this error message:

J(): 3900 unable to allocate real <tmp>[833945,171]
_gsem_eval_chol__loop(): - function returned error
_gsem_eval_chol__work(): - function returned error
_gsem_eval_chol(): - function returned error
mopt__calluser_v(): - function returned error
opt__eval_nr_v2(): - function returned error
opt__eval(): - function returned error
opt__step(): - function returned error
opt__singularH_m_marquardt(): - function returned error
opt__looputil_step(): - function returned error
opt__loop_nr(): - function returned error
opt__loop(): - function returned error
_moptimize(): - function returned error
Mopt_maxmin(): - function returned error
<istmt>: - function returned error

Any ideas on what the issue might be and a workaround? Thanks for your help.

[Felix Bittmann]

This sounds like a question for Stata support. For more help, please provide more infos about your system (hardware specs, operating system, etc...).
As a quick workaround you could estimate a linear probability model as such:

mixed outcome_var exposure1_var i.exposure2_var exposure3_var, vce(robust) || clinic: || patient_id:

[Nick Cox]

You're asking for a matrix with well over 100 million elements. That's quite big.

. di  833945 * 171
1.426e+08

[Clyde Schechter]

Yes, the matrix being requested is very large and is undoubtedly blowing through the limits of memory that is available or that the OS is willing to allocate at the time. But, why did this only happen after 35 successful iterations? Why wasn't a matrix this large also needed at the outset? I'm not aware of anything that makes the storage requirements to estimate the -melogit- likelihood maximum grow during the calculations. So I'm eager to see the answer here.

[Jeff Pitblado (StataCorp)]

Given that comb(19,2)=171, this suggests to me that with the 2 random effect and 2 continuous predictors, the factor variable exposure2_var has 19-2-2-1=14 levels, and the melogit code is having trouble allocating memory for intermediate calculations for the second order derivatives. While the default analytical derivative code for the gradient vector and Hessian matrix are faster, compared to numerical derivatives, they tend to require large amounts of memory.

Try option dnumerical to get melogit to compute the gradient and Hessian matrix numerically. This will take more time, but requires much less memory.