Multiple imputation estimate excluding some imputations

[Garance Delagneau]

Hi everyone,

I would like to run the following code on my imputed dataset:

foreach var in $logit $mlogit {
    mi estimate: prop `var', over(traj_alcohol) // "tabulate" is not an MI-supported command, use prop instead 
    }

However, when I run it I get the following error message:

mi estimate: no observations in some imputations
    This is not allowed.  To identify offending imputations, you can use mi xeq to run the command on individual imputations or you can reissue the
    command with mi estimate, noisily

So I've run the xeq command and there is an issue with observation/imputation number 18. Is there a code I could use to exclude this observation from my estimation?

Thank you
Garance

[Clyde Schechter]

Let's say, for sake of illustration, there are 50 imputations, and only #18 is a problem. Then:

    mi estimate, imputations(1/17 19/50): prop `var', over(traj_alcohol)

will exclude #18.

[daniel klein]

The better question to ask might be: Why is there something wrong with some imputed datasets?

[Garance Delagneau]

Thank you very much, Clyde! The code worked well - although it led to another error message for the next line of code.

This is the code I used

foreach var in $logit $mlogit {
    qui mi estimate, noisily imputations(5/17 19/50): logit `var' i.traj_alcohol
    qui mi test imputations(5/17 19/50) 1.traj_alcohol 2.traj_alcohol 3.traj_alcohol 4.traj_alcohol 5.traj_alcohol
    di "`var'" _col(25) "p=" %4.3f r(p)
    }

This is the error I got:

.         *** imputations 1, 2, 3, 4 and 18 removed because no observations 
. // This will give you MI estimated proportions.
. foreach var in $logit $mlogit {
  2.         qui mi estimate, noisily: logit `var' i.traj_alcohol
  3.         qui mi test 1.traj_alcohol 2.traj_alcohol 3.traj_alcohol 4.traj_alcohol 5.traj_alcohol
  4.         di "`var'" _col(25) "p=" %4.3f r(p)
  5.         }
estimation sample varies between m=1 and m=5; click here for details
r(459);

end of do-file

r(459);

I tried to exclude the observations I excluded in the first line of code, but it didn't change anything.

If you have some time, could you help me with this, please?


I am aware of these three situations to check, but I'm not sure how to check them. I'm using my whole sample (I don't have subsamples)
1. You are fitting a model on a subsample that changes from one imputation to another. For example, you specified the if
expression containing imputed variables.

2. Variables used by model-specific estimators contain values varying across imputations. This results in different sets
of observations being used for completed-data analysis.

3. Variables used in the model (specified directly or used indirectly by the estimator) contain missing values in sets of
observations that vary among imputations. Verify that your mi data are proper and, if necessary, use mi update to
update them.


Note. I get this message "Numbers of observations in e(_N) vary among imputations." at the bottom of each table generated with the previous line of code. Is that something I should get?

CODE

oreach var in $logit $mlogit {
    mi estimate, imputations(5/17 19/50): prop `var', over(traj_alcohol) // "tabulate" is not an MI-supported command, use prop instead 
    }

LAST TWO TABLES OF THE OUTPUT

ultiple-imputation estimates     Imputations     =         45
Proportion estimation             Number of obs   =        699
                                  Average RVI     =     0.0052
                                  Largest FMI     =     0.0104
                                  Complete DF     =        698
DF adjustment:   Small sample     DF:     min     =     687.68
                                          avg     =     691.97
Within VCE type:     Analytic             max     =     695.95

-------------------------------------------------------------------------------
                              |                                   Normal
                              | Proportion   Std. err.     [95% conf. interval]
------------------------------+------------------------------------------------
antidep_combined@traj_alcohol |
         0 Abstained/control  |   .9366451   .0163685      .9045068    .9687833
          0 Low discontinued  |   .9820781   .0125594      .9574193    1.006737
             0 Low sustained  |   .9269515   .0287178      .8705672    .9833358
     0 Moderate discontinued  |   .9576331   .0239365      .9106361     1.00463
        0 Moderate sustained  |   .9569776   .0148808      .9277608    .9861944
            0 High sustained  |   .8707005   .0696415      .7339661    1.007435
         1 Abstained/control  |   .0633549   .0163685      .0312167    .0954932
          1 Low discontinued  |   .0179219   .0125594      -.006737    .0425807
             1 Low sustained  |   .0730485   .0287178      .0166642    .1294328
     1 Moderate discontinued  |   .0423669   .0239365     -.0046301    .0893639
        1 Moderate sustained  |   .0430224   .0148808      .0138056    .0722392
            1 High sustained  |   .1292995   .0696415     -.0074348    .2660339
-------------------------------------------------------------------------------
Note: Numbers of observations in e(_N) vary among imputations.

Multiple-imputation estimates     Imputations     =         45
Proportion estimation             Number of obs   =        699
                                  Average RVI     =     0.0537
                                  Largest FMI     =     0.2085
                                  Complete DF     =        698
DF adjustment:   Small sample     DF:     min     =     359.97
                                          avg     =     630.35
Within VCE type:     Analytic             max     =     690.78

---------------------------------------------------------------------------------------
                                      |                                   Normal
                                      | Proportion   Std. err.     [95% conf. interval]
--------------------------------------+------------------------------------------------
               parenting@traj_alcohol |
      authoriative Abstained/control  |   .4172844   .0334121      .3516795    .4828894
       authoriative Low discontinued  |   .4175569   .0470762       .325125    .5099888
          authoriative Low sustained  |    .300795   .0532759      .1961432    .4054467
  authoriative Moderate discontinued  |   .3679218    .057422      .2551781    .4806656
     authoriative Moderate sustained  |   .3527604   .0351944       .283659    .4218619
         authoriative High sustained  |   .3358196   .1046815      .1301332    .5415061
     authoritarian Abstained/control  |   .1577934   .0245136      .1096629     .205924
      authoritarian Low discontinued  |   .1991266   .0379852      .1245458    .2737074
         authoritarian Low sustained  |   .1997407   .0453892       .110606    .2888755
 authoritarian Moderate discontinued  |   .1551439   .0430118      .0706943    .2395935
    authoritarian Moderate sustained  |     .22524   .0308752      .1646178    .2858621
        authoritarian High sustained  |   .1456812   .0774712     -.0065152    .2978775
        permissive Abstained/control  |   .2515375   .0293464      .1939163    .3091588
         permissive Low discontinued  |   .1547223    .034523      .0869381    .2225064
            permissive Low sustained  |   .2746765   .0508137      .1748863    .3744666
    permissive Moderate discontinued  |    .347034   .0569103      .2352925    .4587756
       permissive Moderate sustained  |   .2058135    .029768      .1473663    .2642607
           permissive High sustained  |   .2107262   .0941719        .02553    .3959224
        disengaged Abstained/control  |   .1733846   .0256232      .1230736    .2236956
         disengaged Low discontinued  |   .2285942   .0401674      .1497266    .3074618
            disengaged Low sustained  |   .2247878   .0475185      .1314691    .3181065
    disengaged Moderate discontinued  |   .1299002   .0402662       .050838    .2089624
       disengaged Moderate sustained  |   .2161861    .030302      .1566905    .2756817
           disengaged High sustained  |   .3077729   .1026349      .1060952    .5094507
---------------------------------------------------------------------------------------
Note: Numbers of observations in e(_N) vary among imputations.

Thank you very much

[Clyde Schechter]

I'm very confused by your post. Initially you said that you had a problem with just imputation 18. The code I suggested in #2 solved it. Now you are reporting that you actually tried to eliminate several other imputations that were problematic. But the code you show doesn't in fact contain the -imputations()- option that would eliminate them.

Daniel Klein has raised the very important question of why you have problematic imputations. Frankly, if there were just one that is causing problems, I would not be terribly woriried about that. But if there are several, then I think something must have gone very wrong in the creation of these imputations. And I think that you need to go back to that and get that fixed so that you have proper imputations to work with.