mi estimate, saving produces different results with xtlogit

Meagan Docherty

Join Date: Jan 2018
Posts: 7

mi estimate, saving produces different results with xtlogit

04 Jan 2018, 15:41

Sorry for the long post. I am new, and hope this question makes sense and is explained correctly. In Stata 14.2, I'm running mi estimate: xtlogit, and saving the estimates:

Code:

mi estimate, dots saving(intest) esample(esamp) post: xtlogit gunca Wdeal Wzprdel Wzysragg t ///i.black#c.Wdeal i.black#c.Wzprdel i.black#c.Wzysragg Bdeal Bzprdel Bzysragg i.sample i.black ///
i.black#c.Bdeal i.black#c.Bzprdel i.black#c.Bzysragg, i(id_s) re

However, I've noticed that when I recover the estimates later using the following commands, they have changed:

Code:

estimates use intest
estimates replay

The coefficients are not the same. In addition, when I originally run the model, the header says things like "Equal FMI" for Model F test, and indicates a large sample DF adjustment. These things are no longer reported when I recover the saved estimates, and it instead only says "Integration method: mvaghermite."
The original estimates:

Code:

. mi estimate, dots saving(intest) esample(esamp) post: xtlogit gunca Wdeal Wzprdel ///Wzysragg t i.black#c.Wdeal i.black#c.Wzprdel i.black#c.Wzysragg Bdeal Bzprdel Bzysragg i.sample ///
i.black i.black#c.Bdeal i.black#c.Bzprdel i.black#c.Bzysragg, i(id_s) reImputations (50):
  .........10.........20.........30.........40.........50 done

Multiple-imputation estimates                   Imputations       =         50
Random-effects logistic regression              Number of obs     =      4,995

Group variable: id_s                            Number of groups  =        999
Random effects u_i ~ Gaussian                   Obs per group:
                                                              min =          5
Integration points = 12                                       avg =        5.0
                                                              max =          5
                                                Average RVI       =     0.2380
                                                Largest FMI       =     0.3297
DF adjustment:   Large sample                   DF:     min       =     458.72
                                                        avg       =   2,139.91
                                                        max       =   5,481.36
Model F test:       Equal FMI                   F(  15,20159.0)   =      24.12
Within VCE type:          OIM                   Prob > F          =     0.0000

----------------------------------------------------------------------------------
           gunca |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
           Wdeal |   1.466972   .4020053     3.65   0.000     .6786674    2.255277
         Wzprdel |   .1297818   .1510445     0.86   0.390    -.1663253    .4258889
        Wzysragg |   1.257324   .2443876     5.14   0.000     .7781461    1.736502
               t |   .2489118   .0612104     4.07   0.000     .1287118    .3691117
                 |
   black#c.Wdeal |
              1  |   .8533233   .4623327     1.85   0.065    -.0533736     1.76002
                 |
 black#c.Wzprdel |
              1  |   .4275371   .1807275     2.37   0.018     .0731915    .7818828
                 |
black#c.Wzysragg |
              1  |  -.7081883   .2904291    -2.44   0.015    -1.277842   -.1385348
                 |
           Bdeal |   2.769167   .6171847     4.49   0.000     1.558898    3.979436
         Bzprdel |   1.559569   .2721862     5.73   0.000     1.024773    2.094365
        Bzysragg |  -.1106736   .2099039    -0.53   0.598    -.5222855    .3009382
                 |
          sample |
         oldest  |   .3196671   .1878959     1.70   0.089    -.0487792    .6881134
         1.black |   .3550727   .3022817     1.17   0.240       -.2378    .9479454
                 |
   black#c.Bdeal |
              1  |   1.676404   .7645017     2.19   0.028       .17703    3.175778
                 |
 black#c.Bzprdel |
              1  |  -.8649218   .3167695    -2.73   0.007    -1.487421   -.2424225
                 |
black#c.Bzysragg |
              1  |   .5600946   .2499411     2.24   0.025     .0699593     1.05023
                 |
           _cons |  -4.958428   .3143691   -15.77   0.000    -5.574925   -4.341931
-----------------+----------------------------------------------------------------
        /lnsig2u |    .247208   .2599575                     -.2627578    .7571738
-----------------+----------------------------------------------------------------
         sigma_u |   1.131568   .1470797                      .8768855     1.46022
             rho |   .2801658   .0524264                      .1894473    .3932494
----------------------------------------------------------------------------------

And the recovered estimates:

Code:

. estimates use intest

. estimates replay

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
active results
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Random-effects logistic regression              Number of obs     =      4,995
Group variable: id_s                            Number of groups  =        999

Random effects u_i ~ Gaussian                   Obs per group:
                                                              min =          5
                                                              avg =        5.0
                                                              max =          5

Integration method: mvaghermite                 Integration pts.  =         12

                                                Wald chi2(15)     =     460.09
Log likelihood  = -902.62441                    Prob > chi2       =     0.0000

----------------------------------------------------------------------------------
           gunca |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
           Wdeal |   1.306689   .3716408     3.52   0.000     .5782866    2.035092
         Wzprdel |   .2003617   .1413164     1.42   0.156    -.0766134    .4773367
        Wzysragg |   1.253574   .2290109     5.47   0.000     .8047209    1.702427
               t |   .2112262    .050596     4.17   0.000     .1120598    .3103926
                 |
   black#c.Wdeal |
              0  |          0  (empty)
              1  |   .7843317   .4204366     1.87   0.062    -.0397088    1.608372
                 |
 black#c.Wzprdel |
              0  |          0  (empty)
              1  |   .4323564   .1660132     2.60   0.009     .1069765    .7577363
                 |
black#c.Wzysragg |
              0  |          0  (empty)
              1  |  -.8261272    .262063    -3.15   0.002    -1.339761   -.3124931
                 |
           Bdeal |   2.756953   .5503567     5.01   0.000     1.678273    3.835632
         Bzprdel |   1.484126   .2171048     6.84   0.000     1.058609    1.909644
        Bzysragg |  -.0699422    .188472    -0.37   0.711    -.4393405     .299456
                 |
          sample |
       youngest  |          0  (empty)
         oldest  |   .3751893   .1656054     2.27   0.023     .0506087    .6997698
                 |
           black |
              0  |          0  (empty)
              1  |   .4189419    .263487     1.59   0.112    -.0974831    .9353668
                 |
   black#c.Bdeal |
              0  |          0  (empty)
              1  |   1.476672   .6676174     2.21   0.027     .1681658    2.785178
                 |
 black#c.Bzprdel |
              0  |          0  (empty)
              1  |  -.8800992   .2486459    -3.54   0.000    -1.367436   -.3927623
                 |
black#c.Bzysragg |
              0  |          0  (empty)
              1  |   .5593919   .2219713     2.52   0.012     .1243362    .9944476
                 |
           _cons |  -4.839172   .2767139   -17.49   0.000    -5.381521   -4.296823
-----------------+----------------------------------------------------------------
        /lnsig2u |   .0375149   .2574606                     -.4670986    .5421283
-----------------+----------------------------------------------------------------
         sigma_u |   1.018934   .1311677                      .7917186    1.311359
             rho |   .2398809   .0469449                      .1600379    .3432782
----------------------------------------------------------------------------------
LR test of rho=0: chibar2(01) = 30.61                  Prob >= chibar2 = 0.000

Does anyone know what is going on or why this happens? Has anyone experienced something similar? Everything is updated, and I ran this multiple times to make sure, but it happened every time.

Tags: None

Meagan Docherty

Join Date: Jan 2018
Posts: 7

04 Jan 2018, 15:50

I played around some more, and found that using the estimates save command separately after mi estimate seemed to reproduce the original results, although I'm still not sure why that's different than the saving() suboption of mi estimate.

Code:

. mi estimate, eform dots saving(test3) esample(newsamp) post: xtlogit gunca Wdeal Wzprdel ///Wzysragg t i.black#c.Wdeal i.black#c.Wzprdel i.black#c.Wzysragg Bdeal Bzprdel Bzysragg i.sample ///
i.black i.black#c.Bdeal i.black#c.Bzprdel i.black#c.Bzysragg, i(id_s) re . estimates save test4
. estimates use test4
. estimates replay

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
active results
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Random-effects logistic regression              Number of obs     =      4,995
Group variable: id_s                            Number of groups  =        999

Random effects u_i ~ Gaussian                   Obs per group:
                                                              min =          5
                                                              avg =        5.0
                                                              max =          5

Integration method: mvaghermite                 Integration pts.  =         12

                                                Wald chi2(.)      =          .
Log likelihood  =          .                    Prob > chi2       =          .

----------------------------------------------------------------------------------
           gunca |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
           Wdeal |   1.466972   .4020053     3.65   0.000     .6786674    2.255277
         Wzprdel |   .1297818   .1510445     0.86   0.390    -.1663253    .4258889
        Wzysragg |   1.257324   .2443876     5.14   0.000     .7781461    1.736502
               t |   .2489118   .0612104     4.07   0.000     .1287118    .3691117
                 |
   black#c.Wdeal |
              0  |          0  (omitted)
              1  |   .8533233   .4623327     1.85   0.065    -.0533736     1.76002
                 |
 black#c.Wzprdel |
              0  |          0  (omitted)
              1  |   .4275371   .1807275     2.37   0.018     .0731915    .7818828
                 |
black#c.Wzysragg |
              0  |          0  (omitted)
              1  |  -.7081883   .2904291    -2.44   0.015    -1.277842   -.1385348
                 |
           Bdeal |   2.769167   .6171847     4.49   0.000     1.558898    3.979436
         Bzprdel |   1.559569   .2721862     5.73   0.000     1.024773    2.094365
        Bzysragg |  -.1106736   .2099039    -0.53   0.598    -.5222855    .3009382
                 |
          sample |
         oldest  |   .3196671   .1878959     1.70   0.089    -.0487792    .6881134
         1.black |   .3550727   .3022817     1.17   0.240       -.2378    .9479454
                 |
   black#c.Bdeal |
              0  |          0  (omitted)
              1  |   1.676404   .7645017     2.19   0.028       .17703    3.175778
                 |
 black#c.Bzprdel |
              0  |          0  (omitted)
              1  |  -.8649218   .3167695    -2.73   0.007    -1.487421   -.2424225
                 |
black#c.Bzysragg |
              0  |          0  (omitted)
              1  |   .5600946   .2499411     2.24   0.025     .0699593     1.05023
                 |
           _cons |  -4.958428   .3143691   -15.77   0.000    -5.574925   -4.341931
-----------------+----------------------------------------------------------------
        /lnsig2u |    .247208   .2599575                     -.2627578    .7571738
-----------------+----------------------------------------------------------------
         sigma_u |   1.131568   .1470797                      .8768855     1.46022
             rho |   .2801658   .0524264                      .1894473    .3932494
----------------------------------------------------------------------------------
LR test of rho=0: chibar2(01) = .                      Prob >= chibar2 =     .

Comment

daniel klein

Join Date: Mar 2014

Posts: 3573
#3

04 Jan 2018, 15:51

This is arguably not well documented. When you save estimates from analysis of multiply imputed data [Edit: with the saving() option], Stata saves the estimates from each of the M imputed datasets, separately, and the final/combined estimates that mi estimate displays. The final/combined estimates are saved as the M+1 result. What you see are the results for the first imputed dataset. What you want is

Code:

estimates use name , number(M+1) estimates replay

Best
Daniel

Last edited by daniel klein; 04 Jan 2018, 15:55.
Comment
Meagan Docherty

Join Date: Jan 2018

Posts: 7
#4

05 Jan 2018, 08:53

Perfect. Thank you so much for your response, Daniel! I've been using the mimrgns command - something else I should thank you for - with the using option. It appears that mimrgns is using the correct, combined estimates when I do this, without having to specify M+1. Is that correct?
For example:

Code:

. mimrgns if black==0 using intest, esample(esamp) at(Bzysragg=.56 Wzysragg=-.18 t=(-2(1)2) Bdeal=0 Wdeal=0 sample=7 Bzprdel=0 Wzprdel=0) cmdmargins post predict(default) dots
Comment
daniel klein

Join Date: Mar 2014

Posts: 3573
#5

05 Jan 2018, 09:45

Originally posted by Meagan Docherty View Post

It appears that mimrgns is using the correct, combined estimates when I do this, without having to specify M+1. Is that correct?

Thanks for the interest in mimrgns (SSC, I suppose). Technically, mimrgns does not use the combined estimates (M+1) at all. It instead uses all of the other M estimates. But the answer you are probably looking for is: Yes, mimrgns does the correct thing when called with the using syntax. You do not need to specify which of the results you want. In fact, mimrgns should not work with results stored as you suggested in #2.

Best
Daniel
Comment
Meagan Docherty

Join Date: Jan 2018

Posts: 7
#6

05 Jan 2018, 10:12

That makes sense. Thank you again for your response!
Comment

Announcement

mi estimate, saving produces different results with xtlogit

Comment

Comment

Comment

Comment

Comment