[Roman Mostazir]

I don't see the Stata is doing anything wrong. You are comparing (post#13) REML estimation from Stata version-11 vs. ML estimation from Stata-version 13. Two are completely different estimation process and naturally will differ. Unlike ML, in REML, the likelihood function is based only on the distribution of residuals at the random part, while in ML the function is maximized based on both fixed and random part.

ML is negatively biased if you have small sample size, and that is happening here as you have only 14 countries. Since it is not being able to compute the variance, it gives you what would have received from OLS. Thus, they don't differ. REML is a better choice when you have a small number of Cluster.

Since you have a small number of cluster, to avoid bias, I would not rely on the LR test from REML even though it is signficant. I would try fitting a null model first with random effect option and will do a Score/Lagrange multiplier test:

xtset country

xtreg yvariable, re

xttest0

If that produces non-zero level-2 variance, an REML based Mixed model will make sense.

[Carlo Lazzaro]

However, it seems that the problem remains when ML is applied in Stata 11 and Stata 13 (please, see post #11).

[Roman Mostazir]

Hi Carlo, it is not. The two commands are slightly different if you notice

Stata-11:

xtmixed gdpgrowthlong loggdp91  avinvest schoolav avpopg rdtot div ///
share1834 neighborgrowth teahijob || country:, var

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
country: Identity            |
                  var(_cons) |   6.09e-24   7.40e-23      2.74e-34    1.35e-13
-----------------------------+------------------------------------------------
               var(Residual) |   .4484908   .0668608      .3348551    .6006897
------------------------------------------------------------------------------
LR test vs. linear regression: chibar2(01) =     0.00 Prob >= chibar2 = 1.0000

Stata-13:
xtmixed gdpgrowthlong loggdp91  avinvest avpopg schoolav rdtot div ///
share1834 neighborgrowth teahijob || country:, mle variance nostderr

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
country: Identity            |
                  var(_cons) |   1.34e-23          .             .           .
-----------------------------+------------------------------------------------
               var(Residual) |   .4484908          .             .           .
------------------------------------------------------------------------------
LR test vs. linear regression: chibar2(01) =     0.00 Prob >= chibar2 = 1.0000

For 13, the author asked not to estimate the standard error for the random effect parameters with "nostderr" option, therefore, that part is blank. The estimation of the variance is similar for both versions.

[Carlo Lazzaro]

Roman.
I agree with you about var (Residual), but I'm not clear with the difference in var (_cons).
I was not able to reproduce that difference in the following example (in Stata 13 only):

. use "http://www.stata-press.com/data/r13/pig.dta", clear
(Longitudinal analysis of pig weights)

. xtmixed weight week || id:, var

Performing EM optimization: 

Performing gradient-based optimization: 

Iteration 0:   log likelihood = -1014.9268  
Iteration 1:   log likelihood = -1014.9268  

Computing standard errors:

Mixed-effects ML regression                     Number of obs      =       432
Group variable: id                              Number of groups   =        48

                                                Obs per group: min =         9
                                                               avg =       9.0
                                                               max =         9


                                                Wald chi2(1)       =  25337.49
Log likelihood = -1014.9268                     Prob > chi2        =    0.0000

------------------------------------------------------------------------------
      weight |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        week |   6.209896   .0390124   159.18   0.000     6.133433    6.286359
       _cons |   19.35561   .5974059    32.40   0.000     18.18472    20.52651
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
id: Identity                 |
                  var(_cons) |   14.81751   3.124226      9.801716    22.40002
-----------------------------+------------------------------------------------
               var(Residual) |   4.383264   .3163348      3.805112     5.04926
------------------------------------------------------------------------------
LR test vs. linear regression: chibar2(01) =   472.65 Prob >= chibar2 = 0.0000

. xtmixed weight week || id:, var nostd

Performing EM optimization: 

Performing gradient-based optimization: 

Iteration 0:   log likelihood = -1014.9268  
Iteration 1:   log likelihood = -1014.9268  

Mixed-effects ML regression                     Number of obs      =       432
Group variable: id                              Number of groups   =        48

                                                Obs per group: min =         9
                                                               avg =       9.0
                                                               max =         9


                                                Wald chi2(1)       =  25337.49
Log likelihood = -1014.9268                     Prob > chi2        =    0.0000

------------------------------------------------------------------------------
      weight |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        week |   6.209896   .0390124   159.18   0.000     6.133433    6.286359
       _cons |   19.35561   .5974059    32.40   0.000     18.18472    20.52651
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
id: Identity                 |
                  var(_cons) |   14.81751          .             .           .
-----------------------------+------------------------------------------------
               var(Residual) |   4.383264          .             .           .
------------------------------------------------------------------------------
LR test vs. linear regression: chibar2(01) =   472.65 Prob >= chibar2 = 0.0000

[Roman Mostazir]

Well, it seems like the difference is in precession in floating point between two versions, because all we are talking about is a difference between 0.00000000000000000000000609 vs. 0.0000000000000000000000134. I am not sure what the documentation says about floating point difference between v-11 and v-13.

[Carlo Lazzaro]

Roman:
I do not know, too.
Anyway, it was (and possibly still is) an interesting thread (for me, at least).

[Krista Bruns]

Thank you everyone for the helpful replies! So is it or is it not allowed to have singletons in a multilevel analysis? In my case this is only 1 country, but still. And does it then still make sense to include a variable that is the group-level mean of an independent variable of interest?

[Roman Mostazir]

Because of ML estimation, singleton observation with one country should not be worrying. The group level variable can be added as level-2 covariate.