Mixed Effects: Standard Errors calculation failed

[Carlos Toruno]

Hi everybody,

I'm calculating a mixed model for educational achievement but, when I specify three variables (isecf, nina & prekfor6) to have a random slope between schools, I get the following output for the random effects:

* mixed puntaje_estandar $x1 $x2 [pw=wgm] || idsc: $rand, cov(ind) vce(robust) ml pw(bsw) pwscale(effective)

----------------------------------------------------------------------------------------------
Robust
Random-effects | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+---------------------------------------------------------------
idsc: Independent |
var(isecf) | 6.83e-11 . . .
var(nina) | 2.54e-12 . . .
var(prekfor6) | 1333.184 . . .

var(_cons) | 76.44768 . . .
-----------------------------+---------------------------------------------------------------
var(Residual) | 3442.109 . . .
----------------------------------------------------------------------------------------------

Where $x1 $x2 are the first and second level variables, respectively. I have 16 covariates at student level and 11 covariates at school level. I set an independent covariance structure.

I noticed that if I set only two variables to have a random slope, mixed is able to compute the standard error, but if I set three or more variables it fails to do it. My questions are: Do I have a problem with the specification of my model? Are the fixed/random estimates trustly to use? What recommendations do you have? I'm using Stata 14 and the dataset have 1,148 observations clustered in 119 groups.

Thank you.

[Carlo Lazzaro]

Carlos:
what if you select a different type of covariance structure?
I wouild also consider the following warning from -mixed- help file:

If vce(robust) is specified, robust variances are clustered at the highest level in the multilevel model.

[Clyde Schechter]

In addition to Carlo's advice, consider that two of your random slopes are very close to zero. It is very difficult to estimate these slopes as they are near the boundary of parameter space and all sorts of convergence issues arise. I would strongly consider just dropping those random slopes from the model: it looks like they are contributing little or nothing anyway, and they may well be making it harder for Stata to estimate other random effects.

[Carlos Toruno]

Thank you Carlo and Clyde for your responses. I read about the different covariance structures available in Stata and I think that Independent and Unstructured are more close to the education literature I have read. I tried with this two, but when using unstructured an error message appeared saying that the standard error calculation failed, so I keep trying with an independent covariance structure because at least I have coefficient and varaiance estimates. If you could recommend me a document to read about the different types of covariance structures, I'd be greatly thankful with you.

About the vce(robust), I set it that way because my observations (students) are clustered in schools, which is my highest level available, so I thought was right to set it that way.

Finally, I would tried to keep the random slopes that are significantly different to zero as Clyde recommended, I'm using the likelihood ratio test for this. Do you know other way to do so?

Thanks

[Clyde Schechter]

Finally, I would tried to keep the random slopes that are significantly different to zero as Clyde recommended, I'm using the likelihood ratio test for this. Do you know other way to do so?

Well, that is one way to do it, although I generally don't like using significance testing to select model structure. I prefer to look at the impact on model predictions and model fit. In your situation, your best estimate of the variance of the random intercept for nina is 2.54 X 10^-12. In the standard deviation scale, this is about 1.6 X 10^-6. So the implication is that for an idsc that is fully 4 standard deviations above or below the mean, that entity's slope for nina is about 6.4 X 10^-6 away from the average entity's nina slope. Now, I don't know how your variables are scaled or what your average nina slope turned out to be. But assuming that these are typical variables in typical data sets, a difference of 6.4 X 10^-6 is going to be a negligible compared to the average nina slope. And even if it isn't, unless the values of nina itself range so large that 6.4X10^-6 times a typical value of nina is an appreciably large number, you are still talking about slope variation that cannot make a visible impact on your model's predictions and can easily be dropped. Can it even begin to compare with, say the residual error or even the variation in the random intercept, both of which are many orders of magnitude larger in your results. Similar considerations apply to isecf.

So do some back of the envelope calculations, and just figure out whether in practical terms these random slopes can make a meaningful contribution to your model. As I've indicated, it would take some unusual data for that to be the case here.

As for trying to get standard errors, it is unlikely that going to unstructured would be helpful. If anything it would make the problem worse because an unstructured covariance matrix has many more parameters that require estimation. With cov(ind) the number of parameters to be estimated is just the number of random intercepts and slopes. Call that number p. With cov(uns) it is, p*(p-1)/2, which is always larger, and even for fairly small p is much larger. My hunch is that if you eliminate the two random slopes whose variance component estimates are effectively zero, and keep the independent structure, Stata will be able to calculate standard errors for the remaining ones.