How to do siginficance test for random effect (coefficient) of growth curve model

[Alice Nee]

Dear Stata users/expert,

I am estimating a growth curve model with cubic form of age and here is my code and result:

mixed cesd9w ctage1 c.ctage1#c.ctage1 c.ctage1#c.ctage1#c.ctage1 || aid: ctage1 if `f1'==1, cov(un)

Performing EM optimization:

Performing gradient-based optimization:

Iteration 0: log likelihood = -83309.993
Iteration 1: log likelihood = -83295.59
Iteration 2: log likelihood = -83295.541
Iteration 3: log likelihood = -83295.541

Computing standard errors:

Mixed-effects ML regression Number of obs = 31,081
Group variable: aid Number of groups = 9,988

Obs per group:
min = 1
avg = 3.1
max = 4

Wald chi2(3) = 351.81
Log likelihood = -83295.541 Prob > chi2 = 0.0000

--------------------------------------------------------------------------------------------
cesd9w | Coef. Std. Err. z P>|z| [95% Conf. Interval]
---------------------------+----------------------------------------------------------------
ctage1 | .3591828 .0396421 9.06 0.000 .2814856 .4368799
|
c.ctage1#c.ctage1 | -.0561205 .0045555 -12.32 0.000 -.0650492 -.0471918
|
c.ctage1#c.ctage1#c.ctage1 | .0020714 .0001519 13.64 0.000 .0017737 .002369
|
_cons | 4.616075 .0987931 46.72 0.000 4.422444 4.809705
--------------------------------------------------------------------------------------------

------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
aid: Unstructured |
var(ctage1) | .0318016 .0020758 .0279826 .0361417
var(_cons) | 8.674551 .2836199 8.136104 9.248634
cov(ctage1,_cons) | -.3120697 .0214446 -.3541003 -.270039
-----------------------------+------------------------------------------------
var(Residual) | 7.877672 .0976526 7.688582 8.071411
------------------------------------------------------------------------------
LR test vs. linear model: chi2(3) = 4467.41 Prob > chi2 = 0.0000

Note: LR test is conservative and provided only for reference.


How can I test if var(ctage1) with .032 and var(_cons) with 8.67 are significant.

When I read Raudenbush and Bryk's book titled "Hierarchical Linear Models", I found that they presented test results for random effects (Table 6.1 on page165).

I wonder anyone can help me doing this type of test.

Thanks,

Alice

[Clyde Schechter]

You can var(ctage1) by running the model both with and without the random slope for ctage1, storing the estimates, and then running -lrtest- to get a likelihood-ratio test.

Testing var(_cons) should not be done. When you have random slopes, the random intercept variation depends on the centering of the variable involved and, in fact, can be arbitrarily set to any pre-specified value non-negative by choice of a centering point for the variable. So unless there is a particularly meaningful centering you have chosen, this variance component is meaningless.

Let's also consider how your model works. You have specified a random coefficient for the linear term in your model, but not the quadratic or cubic terms. Consequently you are not really specifying random cubic models. You have constrained the quadratic and cubic coefficients to the values shown in the fixed-effects results of the table. Thus as the linear term is allowed to vary over a distribution, with the quadratic and cubic terms held constant, the cesd92-ctage1 relationship changes in a constrained manner.

I don't know what the range of values of ctage1 in your data is, but here's an illustration of how this plays out if it ranges between 0 and 10:

clear*
local cons 4.616075
local linear 0.3591828
local quadratic -0.561205
local cubic 0.0020714

local linear_sd = sqrt(0.0318016)

set obs 101


gen ctage1 = (_n-1)/10

forvalues i = -3/3 {
    local j = `i' + 3
    gen cesd9w_`j' = `cons' + (`linear' + `i'*`linear_sd')*ctage1 ///
        + `quadratic'*ctage1^2 + `cubic'*ctage1^3
    label var cesd9w_`j' "Slope at `i' S.D. from mean"
}

graph twoway line cesd9w_* ctage1, sort

The resulting graph is:

(The vertical axis is the value of cesd9w.) So, you do not get a lot of flexibility or difference out of this random slope in this way. It's a reasonable model, but is it what you had in mind?

Now, while we have this graph, let's look at what the var(_cons) term at the aid level does. Different choices of centering for ctage1 are represented graphically by, in effect, drawing a vertical line at different values of ctage1 and seeing where the curves intersect the line. As you can see, a vertical line at 0 here runs through the intersection point of all the curves: the variance of the aid-level intercept in this case would be 0. But as we move our line farther and farther to the right, the curves are spreading out and the variance of the aid-level intercept grows steadily. In principle, you could make it any value you wanted it to be by choosing a corresponding centering value.

[Alice Nee]

Hi Clyde,

Thank you so much for providing the helpful tips for testing and some extra advice about modelling. Your illustration about how the centering affects the variance of the intercept is very informative.

The original range of my age variable is between 12 and 32 and I center it by subtracting it by 12. So the ctage1 variable in the model ranges between 0 and 20.

I did not specify the random effects for the quadratic and cubic form of age. I only have four waves of data. The confidence interval for the quadratic is very large. If I include the cubic form, the model even would not converge. The literature I read suggest not include the random effects of higher-order forms if the data points are just a few. Am I on the right track?

[Clyde Schechter]

I did not specify the random effects for the quadratic and cubic form of age. I only have four waves of data. The confidence interval for the quadratic is very large. If I include the cubic form, the model even would not converge. The literature I read suggest not include the random effects of higher-order forms if the data points are just a few. Am I on the right track?

Yes, I agree with all of this.

[Alice Nee]

Clyde, thank for your feedback! It helps me stay on the right track. - Alice