How to calculate R-squared for three-level hierarchical models (mixed )?

[Rui Yan]

I have a data set with data at three levels (level 1= time, level 2=firm, level 3=country). I am using the mixed command to account for the nested data structure.

To assess the model fit, I would like to calculate R-squared and adjusted R-squared values, as well as the f-test.

Does anyone know if there is any command / module to calculate R-squared for 3-level models? I highly appreciate any advice! Thank you.

[Clyde Schechter]

R² is not a standard statistic with multilevel models. In part, this is because it is unclear how one would define it. Do you want to look at the proportion of variance explained only by the fixed effects? Or do you want to include that explained by the random effects as well? And, if so, at both levels, or only one of them?

If you have an answer to those questions that is appropriate to your particular situation, you can use the -predict- command (see -help mixed postestimation- and click on the link to -predict- for details of how it can be used after -mixed-) to calculate the predicted linear combination of the fixed effects and whichever of the random effects you need. If random effects are included, you can add them to the linear combination to create a new variable that is the combined predictor. Then -corr- will give you the correlation of the predictor and the observed outcome. Finally, you can square that and you have a proportion of variance explained. I do not know of any way to calculate a statistic analogous to the adjusted R².

[Rui Yan]

Hi Clyde,

Many thanks for your detailed explanation. I am investigating the interaction effect, I try to use R² to explain the fit of model and and use adjusted R² to explain the improvement of model after adding the interaction effect. Under such a situation, do I should use aic and bic rather than R² and adjusted R² ?

Many thanks for your kind help!

Rui

[Clyde Schechter]

Yes, I would recommend AIC or BIC over any version of R² for this purpose. That's because these two statistics include penalization for overfitting induced by the extra degree(s) of freedom in the model with the interactioin. R² alone (whether adjusted or not) does not do that.

[Rui Yan]

Many thanks for your kind advice and generous help Clyde!