Dear Stata Community,
my boss has asked me to check a dataset for a multilevel nature.
The data is nested in 2 categories, but he suspects the level 2 effect to be very small and wants to ignore the grouping effect by only conducting a single-level analysis. My task is to check whether this is alright, or if it is necessary to choose a more complicated multi-level approach.
I have done this a couple of times before and I usually begin by checking on the ICC using one of those two regressions:
xtmixed (Dependent Variable) || (Grouping Variable):, mle
xtreg (Dependent Variable), mle i(Grouping Variable)
Now, here comes my problem.
In college I have learned that those two regressions should yield the same results. I also did that kind of investigation on other datasets in the past, and in fact always got the same coefficients, standard deviations and sigmas. But not this time.
For the present dataset, xtmixed and xtreg deliver very different results. The sizes of the coefficients and the standard deviations significantly vary between the models. This goes even more for the ICC: while the xtmixed command delivers comprehensible sizes for sigma_u and sigma_e, leading to an ICC of about 0.1-0.15 for each dependent variable, the xtreg always delivers sigma_u=0, hence leading to an ICC of 0. Hence, according to xtmixed, there is evidence for a level 2 effect, while according to xtreg, it seems that there is nothing like that at all.
How can it be that the results from those two models are so far off in this case? And how do I know which result I can trust - are there any tests I could run? In the past, both of them have always led to the same results, and I can't explain those big differences which I got now.
Thanks to you in advance - any advice will be appreciated!
Best,
Mitja Kleczka
my boss has asked me to check a dataset for a multilevel nature.
The data is nested in 2 categories, but he suspects the level 2 effect to be very small and wants to ignore the grouping effect by only conducting a single-level analysis. My task is to check whether this is alright, or if it is necessary to choose a more complicated multi-level approach.
I have done this a couple of times before and I usually begin by checking on the ICC using one of those two regressions:
xtmixed (Dependent Variable) || (Grouping Variable):, mle
xtreg (Dependent Variable), mle i(Grouping Variable)
Now, here comes my problem.
In college I have learned that those two regressions should yield the same results. I also did that kind of investigation on other datasets in the past, and in fact always got the same coefficients, standard deviations and sigmas. But not this time.
For the present dataset, xtmixed and xtreg deliver very different results. The sizes of the coefficients and the standard deviations significantly vary between the models. This goes even more for the ICC: while the xtmixed command delivers comprehensible sizes for sigma_u and sigma_e, leading to an ICC of about 0.1-0.15 for each dependent variable, the xtreg always delivers sigma_u=0, hence leading to an ICC of 0. Hence, according to xtmixed, there is evidence for a level 2 effect, while according to xtreg, it seems that there is nothing like that at all.
How can it be that the results from those two models are so far off in this case? And how do I know which result I can trust - are there any tests I could run? In the past, both of them have always led to the same results, and I can't explain those big differences which I got now.
Thanks to you in advance - any advice will be appreciated!
Best,
Mitja Kleczka
Comment