J(): 3900 unable to allocate real <tmp>[22901,24090]

[Rochelle Zhang]

Dear Stata listers,

I was running a xtologit model (ordered logit) for a sample of about 21,000 observations, the dependent variable takes on 1,2,3,4 values, and I have 9 continuous variables plus a year dummy , after I issue this command, I got the following error, could you help diagnose the problem and suggest some solutions?

Warm regards,

J(): 3900 unable to allocate real <tmp>[22901,24090]
_gsem_eval_ordinal(): - function returned error
_gsem_eval_iid__obs(): - function returned error
_gsem_eval_iid__wrk(): - function returned error
_gsem_eval_iid(): - function returned error
mopt__calluser_v(): - function returned error
opt__eval_nr_v2(): - function returned error
opt__eval(): - function returned error
opt__looputil_iter0_common(): - function returned error
opt__looputil_iter0_nr(): - function returned error
opt__loop_nr(): - function returned error
_moptimize(): - function returned error
_gsem_build__start_fixed(): - function returned error
_gsem_build__start(): - function returned error
_gsem_build(): - function returned error
_gsem_parse(): - function returned error
st_gsem_parse(): - function returned error
<istmt>: - function returned error


-Rochelle

[Maarten Buis]

Sounds like you included an interaction effect using the factor variable notation and forgot that the default is to treat variables as categorical so you ended up with a model with way too many indicator variables. The way to solve that is to specify e.g. c.varname##i.year, i.e. not to forget to put the c. in front of the variable name you want to treat as continuous.

[Rochelle Zhang]

Thank you Maarten very much !

If I understand you correctly, you suggest I do

xtologit y c.x1 c.x2 c.x3 c.x4 c.x5 c.x6 c.x7 c.x8 c.x9 i.year

x1~x9 are continuous variables, so I must add c prefix , is this correct?

2. since my data is panel data, I must xtset
xtset firmID year

to my knowledge, no fixed effect option is available in xtologit, so the model I use (see above) is just to control for time effect, is there a way to add firm effect?

thanks again,

Rochelle

[Rochelle Zhang]

To follow up, I looked up stata13 manual on xtologit
http://www.stata.com/manuals13/xtologit.pdf

page 4, example, they did not use c. prefix for thk (continuous variable).

Basically, in my last post, I want to ask

1. if I need to use c.prefix for all continuous variable in xtologit.

2. is there a way to do fixed effect in xtologit

-Rochelle

[Maarten Buis]

If I understand you correctly, you suggest I do

xtologit y c.x1 c.x2 c.x3 c.x4 c.x5 c.x6 c.x7 c.x8 c.x9 i.year

That is not necessary. Since you did not give us the complete command in your first post, I had to guess what was going on. My guess was that you typed:

xtologit y x1##year x2 x3

x2 and x3 are correctly specified (i.e. treated as continuous) but x1 will be treated as categorical since it part of an interaction. So if you include interactions you need to type:

xtologit y c.x1##year x2 x3

Since you did not include interactions at all, you can leave the c. prefix away completely.

This means your question is still unsolved. Can you give us the exact command you typed?

[Rochelle Zhang]

I typed
xtologit y x1 x2 x3 x4 x5 x6 x7 x8 i.year

x1~x8 are continuous

x2 is an interaction of x1 and a dummy variable, I did gen x2=x1*D_dummy before I issue xtologit command.

[Richard Williams]

I don't know that it will help any, but usually you include the lower-level terms used to compute an interaction (you have left out D_dummy); and, you let Stata generate the interaction via the use of factor variable notation. So, I think your command should be rewritten as

xtologit y x1 i.D_dummy i.D_dummy#c.x1 x3 x4 x5 x6 x7 x8 i.year

If that doesn't solve the problem (and I doubt that it will) I suggest you simply the model and then build it up, e.g. start with x1 only, then add another variable, then another, etc. You might discover, for example, that things work fine until you add x6, and then you can try to figure out if there is some sort of problem with x6.

[Jeff Pitblado (StataCorp)]

Short reply

Rochelle can use the dnumerical option with xtologit to reduce
its memory needs at a speed cost.

dnumerical is an undocumented option of xtologit that was
officially added to gsem in the 07oct2013 update to Stata 13.

Longer reply

xtologit uses gsem to fit random effects ordinal logistic
models via maximum likelihood.

The analytical derivatives for the gradient vector and Hessian matrix
implemented in gsem are built up from intermediate calculations that
involve matrices with as many rows as the dataset and columns that depend the
model parameters. The largest intermediate matrices are part of the Hessian
calculation.

If \(k\) is the number of model parameters, then the number of unique
elements in the Hessian matrix is

\[
{{k+1}\choose 2} = \frac{(k+1)k}{2}
\]

Rochelle indicated the outcome variable has 4 levels, so there are 4-1=3 cut
points. Rochelle also mentioned that there were 9 continuous predictors and
an unspecified number of year dummies. It appears that Rochelle has 207 year
dummies, because if

\[
k = 3 + 9 + 207 = 219
\]

then

\[
{{219+1}\choose 2} = 24,090
\]

which I got from Mata trace-log that Rochelle reported.

The dnumerical option causes gsem to compute the gradient vector
and Hessian matrix numerically instead of analytically. The result of this is
less memory required but slower model fit.

[Richard Williams]

Rochelle, does year really have more than 200 values??? I wonder how many firms have been around 200+ years. I would double-check the coding for year if 200+ does not seem right.

[Richard Williams]

This is old but probably still relevant. If you are desperate for fixed effects perhaps you could dichotomize the dv and use xtlogit. http://www.stata.com/statalist/archi.../msg00121.html

[Rochelle Zhang]

Many thanks to Richard and Jeff, I will look into the suggestions you all made.

In response post #9 by Richard, my year ranges from 1990 to 2012.

Best,

Rochelle

[Richard Williams]

Jeff thinks year has 200+ values while Rochelle claims it is more like 20. I suggest running a frequencies on year to see who is correct. If, say, year and firmid had accidentally gotten reversed, that might go a long way towards explaining the problems.

[Rochelle Zhang]

Hello, everyone

I try to refit my model using the dnumerical as suggested by Jeff, and also present the summary stats on fyear (time var) below, 1992-2012, so it is not 200+ value. i still got error :

my model
xtologit yvar x1 domi#c.x1 x2 x3 x4 x5 x6 x7 x8 i.fyear domi_u hhi_sic3 a_follow lgta tdebtr eissd prmargin m_seg i.fyear, dnumerical

domi is a binary , x5 and x8 are binary . sample size: 22,000. panel data
stata 13, memory 32 M


J(): 3900 unable to allocate real <tmp>[22901,24090]
_gsem_eval_ordinal(): - function returned error
_gsem_eval_iid__obs(): - function returned error
_gsem_eval_iid__wrk(): - function returned error
_gsem_eval_iid(): - function returned error
mopt__calluser_v(): - function returned error
opt__eval_nr_v2(): - function returned error
opt__eval(): - function returned error
opt__looputil_iter0_common(): - function returned error
opt__looputil_iter0_nr(): - function returned error
opt__loop_nr(): - function returned error
_moptimize(): - function returned error
_gsem_build__start_fixed(): - function returned error
_gsem_build__start(): - function returned error
_gsem_build(): - function returned error
_gsem_parse(): - function returned error
st_gsem_parse(): - function returned error
<istmt>: - function returned error
r(3900);


. sum fyear,d

Data Year - Fiscal
-------------------------------------------------------------
Percentiles Smallest
1% 1994 1992
5% 1996 1992
10% 1997 1992 Obs 22901
25% 1999 1992 Sum of Wgt. 22901

50% 2002 Mean 2002.67
Largest Std. Dev. 4.488194
75% 2006 2012
90% 2009 2012 Variance 20.14388
95% 2010 2012 Skewness .1126115
99% 2011 2012 Kurtosis 2.082029


could you shed some light on this issue?

thanks!

Rochelle

[Richard Williams]

You are sure domi is binary, right? You also have domi_u in the model. domi should also be in there as it is used in the interaction term. See the suggestions I made in post 7 for simplifying the model. But make really really sure domi is only binary. If not it could be generating hundreds of interaction terms.

[ben earnhart]

stata 13, memory 32 M could cause issues. Just let Stata grab as much memory as it can; limiting it to 32 MB is very small.