predict, residuals relevel() not working?

[Jennifer Hauschildt]

Hello,
I am using StataMP18 on Windows10.

I am trying to test the assumptions of my two-level linear regression model.

In this thread it came to my attention that Stata calculates the same residuals when the relevel option is specified and when it is not.
Can anyone explain to me why that is and what it means? Is the relevel option really the righ way to calculate level 2 residuals?

Any help is highly appreciated!

Here my code and outcomes

mixed F01 x1 x2 x3 || L2:, mle
predict l2res, residuals relevel(L2)
predict l2res2, res
list lfd F01 l2res l2res2 in 1/12, noobs sep(12)
 +--------------------------------------------------------+
  |      lfd                   F01       l2res      l2res2 |
  |--------------------------------------------------------|
  | 10000004          5. Zufrieden   -,1256996   -,1256996 |
  | 10000007          5. Zufrieden   -,3762915   -,3762915 |
  | 10000008          5. Zufrieden    ,5353866    ,5353866 |
  | 10000014     4. Eher zufrieden    -1,15214    -1,15214 |
  | 10000043     6. Sehr zufrieden    ,6140783    ,6140783 |
  | 10000048          5. Zufrieden   -,3397757   -,3397757 |
  | 10000076     4. Eher zufrieden   -1,348396   -1,348396 |
  | 10000095   3. Eher unzufrieden   -1,339205   -1,339205 |
  | 10000102   1. Sehr unzufrieden   -4,373418   -4,373418 |
  | 10000136     4. Eher zufrieden   -,8631215   -,8631215 |
  | 10000137     4. Eher zufrieden   -,2085489   -,2085489 |
  | 10000169     6. Sehr zufrieden    ,6701893    ,6701893 |
  +--------------------------------------------------------+

[Erik Ruzek]

Thanks for putting this into a separate thread. I would like to hear from folks at StataCorp about the functionality of relevel() in predict. I thought it might be helpful to add a little bit more information about residuals in mixed models. The best work I've seen in this area is that of Loy & Hoffman (2014; link), whose HLMdiag package in R is second to none. They distinguish between two major types of residuals in mixed effects (multilevel) models:

1. Empirical Bayes (EB) residuals, which are the default in Stata.
2. Least squares (LS) residuals, which are not given in Stata using the postestimation tools, but can be handy because these depend only on the lowest level of the hierarchy and are accordingly, unconfounded by higher-level residuals. These are suggested for examining residuals against fitted values and for plotting residuals against predictors in the model to identify potential misspecification. If group sizes are small, these residuals can be problematic.

To get LS residuals at the lowest level of the hierarchy, you need to do it manually. Below is using the same data as in the Loy & Hoffman (2014) article to get the EB and LS residuals.

use https://www.bristol.ac.uk/cmm/media/runmlwin/tutorial, clear

mixed normexam standlrt || school:

*Empirical Bayes (EB) residuals
predict l1res, residuals
// equivalent to HLMdiag unstandardized EB level-1 residuals for each observation

predict rstandard, rstandard
// equivalent to HLMdiagn standardized EB level-1 residuals

predict fitted, fitted
// equivalent to HLMdiag EB level-1 fitted values

predict l2res, reffect
// equivalent to HLM diag EB level-2 residuals

** The least squares (LS) residuals are calculated by running OLS on each group and then getting the residuals from each group's model
frame create residuals
* Note the use of frameappend, below, downloadable from SSC (ssc install frameappend)
forvalues n = 1/65 {
    frame put normexam standlrt school student if school==`n', into(sch`n')
    frame sch`n': reg normexam standlrt
    frame sch`n': predict ls_resid, residuals
    frame sch`n': predict ls_std, rstandard
    frame sch`n': predict ls_fitted, fitted
    frame residuals: frameappend sch`n', drop
}

frlink 1:1 student school, frame(residuals)
frget ls*, from(residuals)

*compare to tables in https://cran.r-project.org/web/packages/HLMdiag/vignettes/hlm_resid.html
list l1res rstandard fitted ls_resid ls_std ls_fitted in 1/10

Loy & Hoffman (2014) demonstrate LS residuals at the higher level as well, but this would need to be coded up. They also mention two additional types of residuals - semi-standardized and marginal residuals. I did not code these up, but they could be useful. The former for checking for homoskedasticity and the latter for assessing the marginal covariance structure in longitudinal data.

I would love to see these either part of the official mixed postestimation suite or perhaps incorporated into a user-written program by one of the incredible Stata coders who use mixed models and frequent this list regularly.