Different results xtmelogit vs. melogit

[Michael Mazur]

Dear all,

This is my first post. I read the FAQ and I hope I am not breaking the rules.

Here are the relevant information:

Stata version: 14.2
Dataset: 276,961 observations with 5 variables

Regression type: Multilevel binary logistic regression (data has 2 levels: individual-level and country-level)
Dependent variable: "1" if surveyed person said yes
Independent variable: some individual questions + country variable (GDP PPP).

My 1. command: xtmelogit teayyopp i.knowent i.opport c.gdpppp || country:

Result:
------------------------------------------------------------------------------------------

Refining starting values:

Iteration 0: log likelihood = -64895.368 (not concave)
Iteration 1: log likelihood = -64874.887
Iteration 2: log likelihood = -64838.258

Performing gradient-based optimization:

Iteration 0: log likelihood = -64838.258
Iteration 1: log likelihood = -64836.383
Iteration 2: log likelihood = -64836.306
Iteration 3: log likelihood = -64836.305

Mixed-effects logistic regression Number of obs = 276,961
Group variable: country Number of groups = 40

Obs per group:
min = 1,597
avg = 6,924.0
max = 59,515

Integration points = 7 Wald chi2(3) = 10281.36
Log likelihood = -64836.305 Prob > chi2 = 0.0000

------------------------------------------------------------------------------
teayyopp | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.knowent | 1.28444 .0159714 80.42 0.000 1.253137 1.315744
1.opport | .7833621 .0159377 49.15 0.000 .7521248 .8145995
gdpppp | -8.31e-06 3.63e-06 -2.29 0.022 -.0000154 -1.20e-06
_cons | -3.298117 .1416832 -23.28 0.000 -3.575811 -3.020423
------------------------------------------------------------------------------

------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
country: Identity |
sd(_cons) | .3923647 .0450175 .313349 .4913054
------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 3352.58 Prob >= chibar2 = 0.0000

--------------------------------------------------------------------------------------------------------------------------

My 2nd command: melogit teayyopp i.knowent i.opport c.gdpppp || country:

--------------------------------------------------------------------------------------------------------------------------

Fitting fixed-effects model:

Iteration 0: log likelihood = -72022.73
Iteration 1: log likelihood = -66570.714
Iteration 2: log likelihood = -66512.772
Iteration 3: log likelihood = -66512.595
Iteration 4: log likelihood = -66512.595

Refining starting values:

Grid node 0: log likelihood = -66712.026

Fitting full model:

Iteration 0: log likelihood = -66712.026 (not concave)
Iteration 1: log likelihood = -66708.837 (not concave)
Iteration 2: log likelihood = -66707.454 (backed up)
Iteration 3: log likelihood = -66706.215
Iteration 4: log likelihood = -66700.338
Iteration 5: log likelihood = -66700.313
Iteration 6: log likelihood = -66700.313

Mixed-effects logistic regression Number of obs = 276,961
Group variable: country Number of groups = 40

Obs per group:
min = 1,597
avg = 6,924.0
max = 59,515

Integration method: mvaghermite Integration pts. = 7

Wald chi2(3) = 12577.99
Log likelihood = -66700.313 Prob > chi2 = 0.0000
------------------------------------------------------------------------------
teayyopp | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
|
1.knowent | 1.252758 .0157585 79.50 0.000 1.221872 1.283644
1.opport | .9134991 .0154966 58.95 0.000 .8831264 .9438718
gdpppp | -.0000143 5.19e-07 -27.61 0.000 -.0000154 -.0000133
_cons | -3.09413 .0219857 -140.73 0.000 -3.137221 -3.051039
-------------+----------------------------------------------------------------
country |
var(_cons)| .181636 .0816026 .0752986 .4381439
------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 0.00 Prob >= chibar2 = 1.0000

------------------------------------------------------------------------------------------------------------------------

So from my understanding I should not use xtmelogit since this is an outdated command.

When I use melogit, you can see a test of whether the current mixed-effects model represents a significant improvement in fit relative to a standard binary logistic regression. (Prob >= chibar2 =1.0000). With xtmelogit I have 0.000***.

How is it possible that Stata shows such different results even though the difference between xtmelogit and melogit should not be that high? My coefficents are more or less also very similar.

I hope you can help me.

Best wishes,
Michael

[Michael Mazur]

And this is my result when I use this command:

. meqrlogit teayyopp i.knowent i.opport c.gdpppp || country:

Refining starting values:

Iteration 0: log likelihood = -64895.368 (not concave)
Iteration 1: log likelihood = -64874.887
Iteration 2: log likelihood = -64838.258

Performing gradient-based optimization:

Iteration 0: log likelihood = -64838.258
Iteration 1: log likelihood = -64836.383
Iteration 2: log likelihood = -64836.306
Iteration 3: log likelihood = -64836.305

Mixed-effects logistic regression Number of obs = 276,961
Group variable: country Number of groups = 40

Obs per group:
min = 1,597
avg = 6,924.0
max = 59,515

Integration points = 7 Wald chi2(3) = 10281.36
Log likelihood = -64836.305 Prob > chi2 = 0.0000

------------------------------------------------------------------------------
teayyopp | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.knowent | 1.28444 .0159714 80.42 0.000 1.253137 1.315744
1.opport | .7833621 .0159377 49.15 0.000 .7521248 .8145995
gdpppp | -8.31e-06 3.63e-06 -2.29 0.022 -.0000154 -1.20e-06
_cons | -3.298117 .1416832 -23.28 0.000 -3.575811 -3.020423
------------------------------------------------------------------------------

------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
country: Identity |
var(_cons) | .1539501 .0353265 .0981876 .241381
------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 3352.58 Prob >= chibar2 = 0.0000

[Andrew Musau]

From -help xtmelogit-

xtmelogit has been renamed to meqrlogit. xtmelogit continues to work but, as of
Stata 13, is no longer an official part of Stata. This is the original help file,
which we will no longer update, so some links may no longer work.

So I don't see anything inconsistent in the outputs of the commands. However,

LR test vs. logistic model: chibar2(01) = 0.00 Prob >= chibar2 = 1.0000

implies that Stata is unable to do the test for some reason, so you should not rely on this as a result. Try doing the test using lrtest and see what you get:

webuse bangladesh, clear
meqrlogit c_use urban age child* || district:
est sto one
logit c_use urban age child*
est sto two
lrtest one two, force

Res.:

. webuse bangladesh, clear
(Bangladesh Fertility Survey, 1989)

. meqrlogit c_use urban age child* || district:
note: children omitted because of collinearity

Refining starting values:

Iteration 0:   log likelihood = -1219.2682  
Iteration 1:   log likelihood = -1209.3544  
Iteration 2:   log likelihood = -1207.1898  

Performing gradient-based optimization:

Iteration 0:   log likelihood = -1207.1898  
Iteration 1:   log likelihood = -1206.8323  
Iteration 2:   log likelihood = -1206.8322  
Iteration 3:   log likelihood = -1206.8322  

Mixed-effects logistic regression               Number of obs     =      1,934
Group variable: district                        Number of groups  =         60

                                                Obs per group:
                                                              min =          2
                                                              avg =       32.2
                                                              max =        118

Integration points =   7                        Wald chi2(5)      =     109.60
Log likelihood = -1206.8322                     Prob > chi2       =     0.0000

------------------------------------------------------------------------------
       c_use |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       urban |   .7322764   .1194857     6.13   0.000     .4980887    .9664641
         age |  -.0264982   .0078916    -3.36   0.001    -.0419654   -.0110309
      child1 |   1.116002   .1580921     7.06   0.000     .8061466    1.425856
      child2 |   1.365895   .1746691     7.82   0.000      1.02355     1.70824
      child3 |   1.344031   .1796549     7.48   0.000      .991914    1.696148
    children |          0  (omitted)
       _cons |   -1.68929   .1477592   -11.43   0.000    -1.978892   -1.399687
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
district: Identity           |
                  var(_cons) |   .2156188   .0733234      .1107202    .4199007
------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 43.39       Prob >= chibar2 = 0.0000

. est sto one

. logit c_use urban age child*

note: children omitted because of collinearity
Iteration 0:   log likelihood = -1295.4547  
Iteration 1:   log likelihood = -1229.0394  
Iteration 2:   log likelihood = -1228.5266  
Iteration 3:   log likelihood = -1228.5263  
Iteration 4:   log likelihood = -1228.5263  

Logistic regression                             Number of obs     =      1,934
                                                LR chi2(5)        =     133.86
                                                Prob > chi2       =     0.0000
Log likelihood = -1228.5263                     Pseudo R2         =     0.0517

------------------------------------------------------------------------------
       c_use |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       urban |   .7969009   .1051856     7.58   0.000     .5907409    1.003061
         age |  -.0238623   .0075397    -3.16   0.002    -.0386399   -.0090848
      child1 |   1.066757    .152055     7.02   0.000     .7687345    1.364779
      child2 |   1.275993   .1670973     7.64   0.000     .9484886    1.603498
      child3 |    1.21437   .1706222     7.12   0.000     .8799566    1.548783
    children |          0  (omitted)
       _cons |  -1.566886   .1262418   -12.41   0.000    -1.814316   -1.319457
------------------------------------------------------------------------------

. est sto two

. lrtest one two, force

Likelihood-ratio test                                 LR chi2(1)  =     43.39
(Assumption: two nested in one)                       Prob > chi2 =    0.0000

[Michael Mazur]

Thank you for your reply. I still get the same result:


************************************************** ************************************************** *****************

. melogit teayyopp i.knowent i.opport c.gdpppp || country:

Fitting fixed-effects model:

Iteration 0: log likelihood = -72022.73
Iteration 1: log likelihood = -66570.714
Iteration 2: log likelihood = -66512.772
Iteration 3: log likelihood = -66512.595
Iteration 4: log likelihood = -66512.595

Refining starting values:

Grid node 0: log likelihood = -66712.026

Fitting full model:

Iteration 0: log likelihood = -66712.026 (not concave)
Iteration 1: log likelihood = -66708.837 (not concave)
Iteration 2: log likelihood = -66707.454 (backed up)
Iteration 3: log likelihood = -66706.215
Iteration 4: log likelihood = -66700.338
Iteration 5: log likelihood = -66700.313
Iteration 6: log likelihood = -66700.313

Mixed-effects logistic regression Number of obs = 276,961
Group variable: country Number of groups = 40

Obs per group:
min = 1,597
avg = 6,924.0
max = 59,515

Integration method: mvaghermite Integration pts. = 7

Wald chi2(3) = 12577.99
Log likelihood = -66700.313 Prob > chi2 = 0.0000
------------------------------------------------------------------------------
teayyopp | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
|
1.knowent | 1.252758 .0157585 79.50 0.000 1.221872 1.283644
1.opport | .9134991 .0154966 58.95 0.000 .8831264 .9438718
gdpppp | -.0000143 5.19e-07 -27.61 0.000 -.0000154 -.0000133
_cons | -3.09413 .0219857 -140.73 0.000 -3.137221 -3.051039
-------------+----------------------------------------------------------------
country |
var(_cons)| .181636 .0816026 .0752986 .4381439
------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 0.00 Prob >= chibar2 = 1.0000

. est sto one

. logit teayyopp i.knowent i.opport c.gdpppp

Iteration 0: log likelihood = -73506.285
Iteration 1: log likelihood = -67610.881
Iteration 2: log likelihood = -66513.015
Iteration 3: log likelihood = -66512.595
Iteration 4: log likelihood = -66512.595

Logistic regression Number of obs = 276,961
LR chi2(3) = 13987.38
Prob > chi2 = 0.0000
Log likelihood = -66512.595 Pseudo R2 = 0.0951

------------------------------------------------------------------------------
teayyopp | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.knowent | 1.250208 .0157396 79.43 0.000 1.219358 1.281057
1.opport | .9225806 .0154305 59.79 0.000 .8923374 .9528238
gdpppp | -.0000144 4.93e-07 -29.10 0.000 -.0000153 -.0000134
_cons | -3.125354 .0213971 -146.06 0.000 -3.167292 -3.083417
------------------------------------------------------------------------------

. est sto two

. lrtest one two,force

Likelihood-ratio test LR chi2(1) = -375.44
(Assumption: two nested in one) Prob > chi2 = 1.0000

.
************************************************** ************************************************** *****************

Could I just not use meqrlogit? Because these results are good...

I dont understand the real difference between meqrlogit and melogit, even though I did a lot of research regarding this topic...

Thanks a lot!!!!!!

[Andrew Musau]

OK, the test is fine. Notice that the melogit coefficients are all but identical to those of logit, but differ from those of meqrlogit. meqrlogit uses QR decomposition whereas melogit uses the original variance components, so here is a case where the estimation method seems to matter (although it is argued that both approaches are equivalent). What happens if you specify strating values from meqrlogit in melogit? Here, we need to rename the column equation names and column names as Stata assigns different names to the respective estimation matrices, so follow the sequence below:

qui melogit teayyopp i.knowent i.opport c.gdpppp || country:
mat c= e(b)
local eqnames: coleq c
local colnames: coln c
meqrlogit teayyopp i.knowent i.opport c.gdpppp || country:
mat b= e(b)
mat coleq b= `eqnames'
mat coln b= `colnames'
melogit teayyopp i.knowent i.opport c.gdpppp || country:, from(b)

[Michael Mazur]

Thank you for your help. I copy & pasted the code 1:1. This is my output:

************************************************** ************************************************** **

. qui melogit teayyopp i.knowent i.opport c.gdpppp || country:
. mat c= e(b)
. local eqnames: coleq c
. local colnames: coln c
. meqrlogit teayyopp i.knowent i.opport c.gdpppp || country:

Refining starting values:

Iteration 0: log likelihood = -64895.368 (not concave)
Iteration 1: log likelihood = -64874.887
Iteration 2: log likelihood = -64838.258

Performing gradient-based optimization:

Iteration 0: log likelihood = -64838.258
Iteration 1: log likelihood = -64836.383
Iteration 2: log likelihood = -64836.306
Iteration 3: log likelihood = -64836.305

Mixed-effects logistic regression Number of obs = 276,961
Group variable: country Number of groups = 40

Obs per group:
min = 1,597
avg = 6,924.0
max = 59,515

Integration points = 7 Wald chi2(3) = 10281.36
Log likelihood = -64836.305 Prob > chi2 = 0.0000

------------------------------------------------------------------------------
teayyopp | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.knowent | 1.28444 .0159714 80.42 0.000 1.253137 1.315744
1.opport | .7833621 .0159377 49.15 0.000 .7521248 .8145995
gdpppp | -8.31e-06 3.63e-06 -2.29 0.022 -.0000154 -1.20e-06
_cons | -3.298117 .1416832 -23.28 0.000 -3.575811 -3.020423
------------------------------------------------------------------------------

------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
country: Identity |
var(_cons) | .1539501 .0353265 .0981876 .241381
------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 3352.58 Prob >= chibar2 = 0.0000

. mat b= e(b)
. mat coleq b= `eqnames'
. mat coln b= `colnames'
. melogit teayyopp i.knowent i.opport c.gdpppp || country:, from(b)

Fitting fixed-effects model:

Iteration 0: log likelihood = -72022.73
Iteration 1: log likelihood = -66570.714
Iteration 2: log likelihood = -66512.772
Iteration 3: log likelihood = -66512.595
Iteration 4: log likelihood = -66512.595

Refining starting values:

Grid node 0: log likelihood = .
Grid node 1: log likelihood = -66850.336
Grid node 2: log likelihood = -66855.722
Grid node 3: log likelihood = -66867.579

Fitting full model:

Iteration 0: log likelihood = -66850.336
Iteration 1: log likelihood = -66701.07
Iteration 2: log likelihood = -66700.33
Iteration 3: log likelihood = -66700.313
Iteration 4: log likelihood = -66700.313

Mixed-effects logistic regression Number of obs = 276,961
Group variable: country Number of groups = 40

Obs per group:
min = 1,597
avg = 6,924.0
max = 59,515

Integration method: mvaghermite Integration pts. = 7

Wald chi2(3) = 12577.93
Log likelihood = -66700.313 Prob > chi2 = 0.0000
------------------------------------------------------------------------------
teayyopp | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
|
1.knowent | 1.252759 .0157585 79.50 0.000 1.221872 1.283645
1.opport | .9134992 .0154966 58.95 0.000 .8831264 .943872
gdpppp | -.0000143 5.19e-07 -27.61 0.000 -.0000154 -.0000133
_cons | -3.094131 .0219864 -140.73 0.000 -3.137223 -3.051038
-------------+----------------------------------------------------------------
country |
var(_cons)| .1816277 .0815988 .0752953 .4381233
------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 0.00 Prob >= chibar2 = 1.0000

[Andrew Musau]

So the starting values make no difference. I guess it is what it is. You can report the meqrlogit results if you wish. Qualitatively, they are no different from the melogit results, i.e., they arrive at the same inference.

[Michael Mazur]

Thank you very much!