mixed model, calculating std. error of random effects

[cherry singhal]

Hello,

I am using Stata 14.
I have an unbalanced panel dataset. All my independent variables are log-transformed. For my dependent variable, I used neglog transformation.
I am running a mixed model (random intercept, random slope). My question is regarding the output of -mixed-. I was not entirely sure how to present output on this forum so I attached the log file (in both .smcl format and as a pdf) that shows the result and estimates (only).
Notice that the standard error of random effects is blank (perhaps it means not calculated?). What could be the reason for this?
I notice that the model did converge. I also did not get any error like "not concave or backed up or standard error calculation has failed. So how do I understand the blank cells? Does it mean the model is not a fit even if it converged? What would the next step in terms diagnosis or correction?
Thanks!

[cherry singhal]

I ran a LRTEST to check the significance of different random slopes. I ran one lrtest between the random intercept version of my model & the full model (random intercept and 5 random slopes) and I get the following error-

. lrtest x0 xfull
Mixed models are not nested
r(498);

What does this error mean? Thanks!

[Marcos Almeida]

Hello Cherry,

With regards to #2, I suggest you make sure the estimations are done under the restricted maximum likelihood - reml - option. The default in Stata is the maximum likelihood - ml - option.

On what concerns #1, there are many questions. Perhaps you could present, a view of the dataset, commands and output under CODE delimiters or - dataex -, as recommended in the FAQ.

Edited, after taking a look on the PDF with the output:

Where we read "Computing standard errors", since the SEs are missing, we should read aftwards "standard error calculation failed".

I wonder if you edited the presentation of the output. That said, and according to the Stata manual on mixed models (http://www.stata.com/manuals13/me.pdf):

A failure to converge can take any one of three forms:
1. repeated nonconcave or backed-up iterations without convergence;
2. a Hessian (second-derivative) calculation that has become asymmetric, unstable, or has missing values; or
3. the message “standard-error calculation has failed” when computing standard errors.

Best,

Marcos

[cherry singhal]

Thanks Marcus for your response.
First, I did not edit the output in any way because that would not help me here. As I mentioned in my original post, I did not get "standard error calculation failed" error.
Second, I shall use dataex from now on. Thanks.
Third, why do you think REML would be a better option compared to MLE? I will run REML just to compare and see if I get the same error or not.

[Joseph Coveney]

Notice that the standard error of random effects is blank (perhaps it means not calculated?). What could be the reason for this?
I notice that the model did converge. I also did not get any error like "not concave or backed up or standard error calculation has failed. So how do I understand the blank cells? Does it mean the model is not a fit even if it converged? What would the next step in terms diagnosis or correction?

My guess is that it's because the estimates for three of your five random slopes and that of your random intercept are zero. Try omitting fitting random slopes for those three and see whether you then get nonzero estimates for the intercept (and get standard errors).

[cherry singhal]

thank you Joseph!
Given your suggestion, I tried building up the model. When I include a single random effect, each one is significant (SE, interval and all, no issues) and also with a subset of random effects, I am good; just not with the full model.

So if the variance of the random effect is too low, the SE is not calculated and this means there is no technical issue with the model equation but just that I do not find evidence for that random effect to be significant?
But do you know why the error - "Mixed models are not nested" when doing an lrtest? The lrtest should just show me that p-val>0.05 and that the null hypothesis is rejected. Why the error? Again, the lrtest was between the random intercept model and the full model.

Thanks again!

[Joseph Coveney]

But do you know why the error - "Mixed models are not nested" when doing an lrtest? The lrtest should just show me that p-val>0.05 and that the null hypothesis is rejected. Why the error? Again, the lrtest was between the random intercept model and the full model.

I think that this behavior has been noted before on the list, and not all that long ago. When mixed is unable to estimate the standard errors for some variances in the full model, it behaves as if the random effects are not included in the model, and so the reduced model doesn't appear to mixed to be nested in the full model. Compare the values of the rank of the e(V) in the full and reduced models shown below.

.ÿversionÿ14.2

.ÿ
.ÿclearÿ*

.ÿsetÿmoreÿoff

.ÿsetÿseedÿ1368688

.ÿquietlyÿsetÿobsÿ500

.ÿforvaluesÿiÿ=ÿ1/6ÿ{
ÿÿ2.ÿÿÿÿÿÿÿÿÿgenerateÿdoubleÿvar`i'ÿ=ÿrnormal()
ÿÿ3.ÿ}

.ÿgenerateÿintÿpidÿ=ÿmod(_n,ÿ100)

.ÿmixedÿvar1-var6ÿ||ÿpid:ÿvar2-var6ÿ,ÿnolrtestÿnolog

Mixed-effectsÿMLÿregressionÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿÿÿÿÿ=ÿÿÿÿÿÿÿÿ500
Groupÿvariable:ÿpidÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿgroupsÿÿ=ÿÿÿÿÿÿÿÿ100

ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿObsÿperÿgroup:
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿminÿ=ÿÿÿÿÿÿÿÿÿÿ5
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿavgÿ=ÿÿÿÿÿÿÿÿ5.0
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿmaxÿ=ÿÿÿÿÿÿÿÿÿÿ5

ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿWaldÿchi2(5)ÿÿÿÿÿÿ=ÿÿÿÿÿÿÿ0.56
Logÿlikelihoodÿ=ÿ-688.20467ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿProbÿ>ÿchi2ÿÿÿÿÿÿÿ=ÿÿÿÿÿ0.9896

------------------------------------------------------------------------------
ÿÿÿÿÿÿÿÿvar1ÿ|ÿÿÿÿÿÿCoef.ÿÿÿStd.ÿErr.ÿÿÿÿÿÿzÿÿÿÿP>|z|ÿÿÿÿÿ[95%ÿConf.ÿInterval]
-------------+----------------------------------------------------------------
ÿÿÿÿÿÿÿÿvar2ÿ|ÿÿ-.0187685ÿÿÿ.0414998ÿÿÿÿ-0.45ÿÿÿ0.651ÿÿÿÿ-.1001066ÿÿÿÿ.0625695
ÿÿÿÿÿÿÿÿvar3ÿ|ÿÿÿ.0003848ÿÿÿ.0469232ÿÿÿÿÿ0.01ÿÿÿ0.993ÿÿÿÿÿ-.091583ÿÿÿÿ.0923527
ÿÿÿÿÿÿÿÿvar4ÿ|ÿÿ-.0146714ÿÿÿ.0457114ÿÿÿÿ-0.32ÿÿÿ0.748ÿÿÿÿ-.1042642ÿÿÿÿ.0749213
ÿÿÿÿÿÿÿÿvar5ÿ|ÿÿÿ.0208349ÿÿÿ.0447597ÿÿÿÿÿ0.47ÿÿÿ0.642ÿÿÿÿ-.0668925ÿÿÿÿ.1085623
ÿÿÿÿÿÿÿÿvar6ÿ|ÿÿÿÿ.008833ÿÿÿ.0430182ÿÿÿÿÿ0.21ÿÿÿ0.837ÿÿÿÿ-.0754811ÿÿÿÿ.0931471
ÿÿÿÿÿÿÿ_consÿ|ÿÿÿ.0019015ÿÿÿ.0494832ÿÿÿÿÿ0.04ÿÿÿ0.969ÿÿÿÿ-.0950838ÿÿÿÿ.0988869
------------------------------------------------------------------------------

------------------------------------------------------------------------------
ÿÿRandom-effectsÿParametersÿÿ|ÿÿÿEstimateÿÿÿStd.ÿErr.ÿÿÿÿÿ[95%ÿConf.ÿInterval]
-----------------------------+------------------------------------------------
pid:ÿIndependentÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(var2)ÿ|ÿÿÿ1.96e-14ÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿ.
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(var3)ÿ|ÿÿÿÿ.028142ÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿ.
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(var4)ÿ|ÿÿÿ.0105775ÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿ.
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(var5)ÿ|ÿÿÿ1.88e-19ÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿ.
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(var6)ÿ|ÿÿÿ1.38e-21ÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿ.
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(_cons)ÿ|ÿÿÿ.0734298ÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿ.
-----------------------------+------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(Residual)ÿ|ÿÿÿÿ.820373ÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿ.
------------------------------------------------------------------------------

.ÿdisplayÿinÿsmclÿasÿtextÿe(rank)
6

.ÿestimatesÿstoreÿFull

.ÿ
.ÿmixedÿvar1-var6ÿ||ÿpid:ÿ,ÿnolrtestÿnolog

Mixed-effectsÿMLÿregressionÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿÿÿÿÿ=ÿÿÿÿÿÿÿÿ500
Groupÿvariable:ÿpidÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿgroupsÿÿ=ÿÿÿÿÿÿÿÿ100

ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿObsÿperÿgroup:
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿminÿ=ÿÿÿÿÿÿÿÿÿÿ5
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿavgÿ=ÿÿÿÿÿÿÿÿ5.0
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿmaxÿ=ÿÿÿÿÿÿÿÿÿÿ5

ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿWaldÿchi2(5)ÿÿÿÿÿÿ=ÿÿÿÿÿÿÿ0.69
Logÿlikelihoodÿ=ÿ-688.83251ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿProbÿ>ÿchi2ÿÿÿÿÿÿÿ=ÿÿÿÿÿ0.9833

------------------------------------------------------------------------------
ÿÿÿÿÿÿÿÿvar1ÿ|ÿÿÿÿÿÿCoef.ÿÿÿStd.ÿErr.ÿÿÿÿÿÿzÿÿÿÿP>|z|ÿÿÿÿÿ[95%ÿConf.ÿInterval]
-------------+----------------------------------------------------------------
ÿÿÿÿÿÿÿÿvar2ÿ|ÿÿ-.0249981ÿÿÿ.0414371ÿÿÿÿ-0.60ÿÿÿ0.546ÿÿÿÿ-.1062133ÿÿÿÿ.0562171
ÿÿÿÿÿÿÿÿvar3ÿ|ÿÿÿ.0023262ÿÿÿ.0433979ÿÿÿÿÿ0.05ÿÿÿ0.957ÿÿÿÿ-.0827322ÿÿÿÿ.0873846
ÿÿÿÿÿÿÿÿvar4ÿ|ÿÿÿ-.017306ÿÿÿ.0443065ÿÿÿÿ-0.39ÿÿÿ0.696ÿÿÿÿ-.1041452ÿÿÿÿ.0695332
ÿÿÿÿÿÿÿÿvar5ÿ|ÿÿÿ.0179594ÿÿÿ.0449118ÿÿÿÿÿ0.40ÿÿÿ0.689ÿÿÿÿ-.0700661ÿÿÿÿÿ.105985
ÿÿÿÿÿÿÿÿvar6ÿ|ÿÿÿÿ.006928ÿÿÿ.0432122ÿÿÿÿÿ0.16ÿÿÿ0.873ÿÿÿÿ-.0777663ÿÿÿÿ.0916224
ÿÿÿÿÿÿÿ_consÿ|ÿÿ-.0012897ÿÿÿ.0495984ÿÿÿÿ-0.03ÿÿÿ0.979ÿÿÿÿ-.0985008ÿÿÿÿ.0959214
------------------------------------------------------------------------------

------------------------------------------------------------------------------
ÿÿRandom-effectsÿParametersÿÿ|ÿÿÿEstimateÿÿÿStd.ÿErr.ÿÿÿÿÿ[95%ÿConf.ÿInterval]
-----------------------------+------------------------------------------------
pid:ÿIdentityÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(_cons)ÿ|ÿÿÿ.0737844ÿÿÿ.0368357ÿÿÿÿÿÿ.0277342ÿÿÿÿ.1962971
-----------------------------+------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(Residual)ÿ|ÿÿÿ.8571534ÿÿÿ.0606433ÿÿÿÿÿÿ.7461675ÿÿÿÿ.9846475
------------------------------------------------------------------------------

.ÿdisplayÿinÿsmclÿasÿtextÿe(rank)
8

.ÿcaptureÿnoisilyÿlrtestÿFull
Mixedÿmodelsÿareÿnotÿnested

.ÿ
.ÿexit

endÿofÿdo-file


.

You can

matrix list e(V)

after the full and reduced models to see things more clearly.

[Marcos Almeida]

With regards to the discussion about selecting ml or reml, polemic as is may, since the fixed-effects specification seemed to me to be the same in both of your models, it is usually indicated the reml option so as to get "unbiased" estimates of the parameters related to the variance.

[cherry singhal]

thank you much Joseph for the elaborate explanation. I also learned a couple of new commands!
As you said, somebody else experienced the same error before on this forum. I wonder if there is a resource that compiles such behaviors/errors (the more frequently asked ones at least) that people like me can access. Stata help manuals are great starting point though.

[cherry singhal]

Thank you Marcos, this is good info to know. The fixed effects do remain the same.