likelihood test with xtmixed

[Maya Lani]

Hi all,

I am estimating a multilevel model, where I use sample weights for the individual level variables.

xtmixed y female age agesqth married kids gini [pw=weight] || country:


I want to compare the model above to a random slopes model, so I use the likelihood test in stata.But I get the following error:

. lrtest rnd m1
LR test likely invalid for models with robust vce
r(498);

When I run the same model without any weights and compare to a random slopes model also without weight specification, then I get no error message with lrtest.

Is it wrong to use the weights in my estimation and then use the option 'force' with lrtest to compute the likelihood ration regardless?

[Andrew Musau]

This may help

https://www.stata.com/support/faqs/s...od-ratio-test/

[Weiwen Ng]

This may help

https://www.stata.com/support/faqs/s...od-ratio-test/

Andrew, that resource explains why the LR test is generally invalid under pweighting. With LR testing of nested random effects models, there's an additional complication that if you are eliminating one of the random effects, you're essentially testing that its variance is zero, which is on the boundary of the parameter space. I think it's been shown theoretically (i.e. by people far smarter than I) that the test statistic is a mixture of chi-square distributions (see the note that applies to the chi-bar test in the bottom of your -mixed- output).

Additionally, I'm not sure if -svy: test- will work given that Maya fit the mixed model without the prefix command. If it does work, I think the syntax would be something like this:

svy: test [lns_1_1_1]_cons = 0

Maya, for your reference, that is the log of the standard deviation of the first random effect in your output. Stata calculates and internally stores that value as the log of the standard deviation, but it automatically exponentiates and squares it to convert it to a variance in your output. You'd be testing that the variance is equal to zero. Also for your reference, -xtmixed- is an old command which still works, but the proper command name is -mixed-. If the command doesn't work with the -svy:- prefix, try omitting the -svy:- prefix.

I honestly do not know if conducting a Wald test on the variance of the random intercept is technically correct. It may be, or it may not be. If the confidence interval for the variance of the random slope (in your case) clearly excludes zero, then honestly, I would just rely on that.

Edit: Maya, the above syntax tests the first random effect in your output, whatever that is. If you want to test the random slope, it's probably the first one, but you should double check. You can list the contents of e(b), then exponentiate and square the cell you want, e.g.

webuse pig
mixed weight week || id:week
matrix list e(b)
e(b)[1,5]
        weight:     weight:   lns1_1_1:   lns1_1_2:    lnsig_e:
          week       _cons       _cons       _cons       _cons
y1   6.2098958   19.355613  -.49974538   .95524242   .23463013

display exp( -.49974538)^2
.36806683

[Andrew Musau]

Thanks Weiwen Ng for the clarification.

[Weiwen Ng]

Thanks Weiwen Ng for the clarification.

I'm honestly not sure it was a clarification. It's more like, I don't know if this is theoretically valid, but here's how you would run the command in Stata...

[Andrew Musau]

I will change that to suggestion then. -svy- is not part of my vocabulary, so I do not have a clue what is the right approach here. I think that your advice is as good as they come.

[Steve Samuels]

xtmixed was replaced by mixed in Stata 13. There is a section on Survey Data in the manual entry, located in the ME manual. A conditional weight is required for each level in the model. For country, that weight can be 1. In many cross-country surveys, an independent sample is drawn in each country, so country would also be sampling stratum. Within each country, there might have been additional sampling stages. If so, the unit for the first stage would be the PSU (Primary Sampling Unit) variable if you svyset and it must also appear as a second level in the multilevel model.

[Maya Lani]

This may help

https://www.stata.com/support/faqs/s...od-ratio-test/

Thanks Andrew, this helps me understand a bit why lrtest does not work with weights

[Maya Lani]

Andrew, that resource explains why the LR test is generally invalid under pweighting. With LR testing of nested random effects models, there's an additional complication that if you are eliminating one of the random effects, you're essentially testing that its variance is zero, which is on the boundary of the parameter space. I think it's been shown theoretically (i.e. by people far smarter than I) that the test statistic is a mixture of chi-square distributions (see the note that applies to the chi-bar test in the bottom of your -mixed- output).

Additionally, I'm not sure if -svy: test- will work given that Maya fit the mixed model without the prefix command. If it does work, I think the syntax would be something like this:

svy: test [lns_1_1_1]_cons = 0

Maya, for your reference, that is the log of the standard deviation of the first random effect in your output. Stata calculates and internally stores that value as the log of the standard deviation, but it automatically exponentiates and squares it to convert it to a variance in your output. You'd be testing that the variance is equal to zero. Also for your reference, -xtmixed- is an old command which still works, but the proper command name is -mixed-. If the command doesn't work with the -svy:- prefix, try omitting the -svy:- prefix.

I honestly do not know if conducting a Wald test on the variance of the random intercept is technically correct. It may be, or it may not be. If the confidence interval for the variance of the random slope (in your case) clearly excludes zero, then honestly, I would just rely on that.

Edit: Maya, the above syntax tests the first random effect in your output, whatever that is. If you want to test the random slope, it's probably the first one, but you should double check. You can list the contents of e(b), then exponentiate and square the cell you want, e.g.

webuse pig
mixed weight week || id:week
matrix list e(b)
e(b)[1,5]
weight: weight: lns1_1_1: lns1_1_2: lnsig_e:
week _cons _cons _cons _cons
y1 6.2098958 19.355613 -.49974538 .95524242 .23463013

display exp( -.49974538)^2
.36806683

Dear Weiwen, I indeed do not use the command "svy" for survey analysis.
I simply want to decide which model is better, and it seems to be wrong to apply the likelihood test in case of sample weights.
What do you exactly mean by "it's probably the first one" in your edit?