Problem with intraclass correlation coefficient in Mixed-effects

[Alejandra Pallamar]

Hi everyone¡

I have cluster data from a School District ( 2000 Workers nested in 25 Schools), I have a continuous dependent variable and an independent variable, and several dichotomous covariates (at the individual and group level)
I am running Mixed in Stata 16. The null model's ICC is low (0.036), but the LR test vs. linear model suggests that the two-level model offers a significantly better fit to the data than the single-level model. Adding the first group level variable (variable at the School level) works the same...meaning that ICC is low. Still, the LR test vs. linear model suggests that the two-level model offers a significantly better fit to the data than the single-level model. When adding most group-level variables the ICC looks very odds (I suppose extremely low: 7.03e-16 ) and the LR test vs. linear model suggests that the two-level model does not offer a significantly better fit to the data than the single-level. My questions:
-How should I interpret an ICC value of: 7.03e-16
-Should I use standard regression and skip clustering the data?


Thanks, ¡¡¡

[Leonardo Guizzetti]

That number is essentially zero, meaning you have accounted for all variability within the model and adding the school level clustering adds nothing to the model. You can proceed with standard regression if you plan to maintain those same covariates.

[Alejandra Pallamar]

Thank you so much for your quick response, Leonardo¡¡. If I change to standard regression should I use some special command if I have cluster data, even with a small ICC, or not?

[Alejandra Pallamar]

The other question, as I aay the null model's ICC is already low (0.036), so I suppose I can just go with standard regression from there......

[Leonardo Guizzetti]

If you switch to regular regression, simply use -regress- instead of -mixed-, and there would be no need to account for the school in your model with ICC=0.

You say the null model has an ICC = 0.036 and that the LR test suggests the mixed model is a better fit versus the fixed effect model. Opinions here differ as to how to account for the school level clustering, if at all. On the one hand, you could ignore it and run a standard fixed effect regression. This model says that schools are completely independent of your outcome. Adding a fixed effect of school to this model (adding -I.school- as a covariate) says that every school adds its own disturbance or effect to the model, and so estimates become conditional on the particular school. On the other extreme is the mixed model, which assumes all schools come from a normal distribution, allowing staff in schools to be correlated withi each other. Again, estimates are conditional on the random school effect.

What you should do requires more knowledge of your domain and what you will use the model for. One thing I will note is that ignoring clustering results in anticonservative standard errors, by a possibly large factor, even with a small ICC but large group size.

[Alejandra Pallamar]

Theoretically, the school effect is important to consider in my topic....but I don't know how to explain such a strange ICC.....Is it possible to present these results and continue in a mixed model????. Or essentially it is necessary to switch to a standard model since it looks like no need to account for the school in the model???

[Leonardo Guizzetti]

Yes, you can still continue with a mixed model. When you get to the model where the ICC becomes essentially zero, then you can present results for a standard regression and note that the ICC from the analogous mixed model was zero. It is possible for the ICC to become zero after including covariates if, for example, one or more of those covariates are strongly correlated with school, so you have already captured that information in the form of other covariate(s), then there will be nothing left to estimate at the school (cluster) level.

[Alejandra Pallamar]

Thank you Leonardo for your guidance, I will do that.