How does Stata determine whether a variance component in a random effects model is significant when using mixed and esttab?

[Christina Padilla]

Hello,

I am running a 2-level mixed model using the mixed command in Stata version 15.0, and I am mainly interested in understanding whether my random effect is significantly different from zero. I understand that you can use the LR test to determine whether one model fits better than another (e.g. is a random slope needed or is a model with just a random intercept needed?), but this is not the same as understanding whether the variance/SD on the random slope differs significantly from zero.
I am using esttab to output my results, and this command will give me *s to indicate when coefficients and variance components are significant at different levels.

My question is this: What test is Stata using to determine the significance of the random slope? In other words, how does Stata decide whether a random slope is significant, and at what alpha level?

use the following code:

mixed p2mathacts tx p1cogstim $covars || ID: tx , mle cov(un) stddev noconstant
est sto m1
mixed p2litacts tx p1cogstim $covars || ID: tx , mle cov(un) stddev noconstant
est sto m2
mixed p2readn tx p1readn $covars || ID: tx , mle cov(un) stddev noconstant
est sto m3
esttab m1 m2 m3, se transform(ln*: exp(@) exp(@)) eqlabels("" "sd(tx)" "sd(Residual)")

Here is the output from my esttab command:

	(1)	(2)	(3)
	p2mathacts	p2litacts	p2readn
tx	0.217***	0.186***	0.101**
	(0.043)	(0.043)	(0.031)
p1cogstim	1.290***	1.367***
	(0.051)	(0.051)
cohort3	-0.0832*	-0.138***	-0.0148
	(0.039)	(0.038)	(0.029)
female	0.0698	0.0830*	0.0907**
	(0.037)	(0.037)	(0.028)
black	0.0137	-0.0585	-0.159***
	(0.054)	(0.054)	(0.041)
hispanic	0.044	-0.0925	-0.120**
	(0.059)	(0.059)	(0.045)
hleng	0.140*	-0.0811	-0.105*
	(0.069)	(0.068)	(0.052)
momage	0.00183	0.0016	0.00463*
	(0.003)	(0.003)	(0.002)
hsless	0.0419	0.0637	-0.0854*
	(0.048)	(0.048)	(0.037)
hs	0.0541	0.0752	-0.0933*
	(0.047)	(0.047)	(0.036)
married	0.039	-0.0261	0.000229
	(0.054)	(0.053)	(0.041)
prevmarried	-0.0423	-0.0513	0.0234
	(0.058)	(0.058)	(0.045)
teenmom	0.0311	-0.00471	0.0491
	(0.055)	(0.055)	(0.042)
momimmig	-0.0556	-0.0898	-0.111*
	(0.063)	(0.064)	(0.048)
bothbio	-0.0683	-0.0026	0.0401
	(0.052)	(0.052)	(0.040)
p1readn			0.328***
			(0.015)
_cons	1.721***	2.201***	1.213***
	(0.125)	(0.124)	(0.088)
sd(tx)
_cons	0.308***	0.316***	0.191***
	(0.034)	(0.034)	(0.028)
sd(Residual)
_cons	1.017	1.005	0.834***
	(0.013)	(0.013)	(0.010)
N	3182	3151	3581
Standard errors in parentheses
*p<0.05, **p<0.01, ***p<0.001

Note that I am not interested in comparing these three models, just outputting multiple models simultaneously to see them next to each other.

The random slope that I am interested in understanding is in bold. How is Stata determining that the .308, .316, and .191 are significant at p<0.001?

[Maarten Buis]

Hello,
My question is this: What test is Stata using to determine the significance of the random slope? In other words, how does Stata decide whether a random slope is significant, and at what alpha level?

What you showed is the output of esttab. This is a very generic program; it is intended to be used by all estimation commands. That is a good thing, but it means it cannot take specifics of individual models into account. In this case the test is a normal wald test, which is not appropriate in this case.

[Christina Padilla]

Hi Maarten,

Thank you very much for your response - this makes sense.

Is there a test you would recommend to assess whether variance parameters are significant? I've heard varying opinions on the LR test as well as assessing whether confidence intervals overlap with zero (which doesn't make sense to me, since a variance could never be negative).