Running contrast pwcompare commands with mimrgns

[Evelyn Chan]

Hello,

I recently learned about mimrgns module that allows you to run margins after an MI Estimate. I'd like to also run pwcompare and contrast, which I would with just a margins command, i.e. margins var, pwcompare(effects). I am having trouble running pwcompare and contrast with mimrgns, any suggestions?

[daniel klein]

Evelyn,

-mimrgns- is (most likely) from SSC, as you are asked to explain.

The -pwcompare- option you want to use was added to the -margins- command in Stata release 13 to which I (as the 'author' of -mimrgns-) unfortunately still do not have permanent access yet. Looking at the online help files however, I strongly suspect that -mimrgns- combines the 'wrong' matrices, namely e(b) and e(V), whereas you probably want to combine e(b_vs) and e(V_vs). This is speculation of course, as you are not very precise about your problem and I have no clear idea what

having trouble running pwcompare and contrast with mimrgns

exactly means. Do you get an error message you do not understand? Do you get an output but not the one you expect? Something else?

I imagine this to be a not so complicated fix, given access to Stata 13. However, it is probably not that easy - and more important: a not even testable 'shot in the dark' - without access to Stata 13. I am afraid it will take me a few weeks to get this done, and you might want to give it a shot yourself in the meantime.

All the basic ideas needed are given here.

Best
Daniel

[Evelyn Chan]

Thanks for the response. I noticed that running, mimrgns (var), pwcompare(effects) doesn't produce pairwise comparisons. Running the command will give the same output as if I ran mimrgns (var). I was just hoping to run contrast, pwcompare with mimrgns.

. mimrgns (var)
Multiple-imputation estimates	Imputations =	20
Predictive margins	Number of obs =	3989
	Average RVI =	0.0736
	Largest FMI =	0.0740
DF adjustment: Large sample	DF: min =	3523.14
	avg =	6021.11
Within VCE type: Delta-method	max =	11286.41
Coef. Std. Err.	t	P>t [95% Conf.	Interval]
var
0 .1756739 .0212105	8.28	0.000 .1340878	.2172599
1 .1756739 .0212105	8.28	0.000 .1340878	.2172599
2 .2259046 .0283255	7.98	0.000 .1703817	.2814275
3 .4791126 .0595425	8.05	0.000 .3623792	.5958459
4 .5667947 .0813287	6.97	0.000 .4073672	.7262222

. mimrgns (var), pwcompare(effects)
Multiple-imputation estimates	Imputations =	20
Pairwise comparisons of predictive margins
Average RVI =	0.0736
Largest FMI =	0.0740
DF adjustment: Large sample	DF: min =	3523.14
avg =	6021.11
Within VCE type: Delta-method	max =	11286.41
Coef. Std. Err. t	P>t [95% Conf.	Interval]
var
0 .1756739 .0212105 8.28	0.000 .1340878	.2172599
1 .1756739 .0212105 8.28	0.000 .1340878	.2172599
2 .2259046 .0283255 7.98	0.000 .1703817	.2814275
3 .4791126 .0595425 8.05	0.000 .3623792	.5958459
4 .5667947 .0813287 6.97	0.000 .4073672	.7262222

[Richard Williams]

Daniel and others, if you are in a mad rush to try Stata 13, you can get a 30 day demo version: http://www.stata.com/customer-service/evaluate-stata/

[pgr451]

I'm having trouble interpretting a mimrgns output. I thought the predicted probability cannot exceed 1? Am I missing a command? Can someone please help me?



mi estimate, esampvaryok post cmdok: logit milch comhigh sexmil tbi ptsd i.racenew2 age sex i.mar2 i.cohab3 i.cursep child edu rank i.serv2 i.deploy relat relsal prayst

Multiple-imputation estimates Imputations = 25
Logistic regression Number of obs = 39,877
Average RVI = 0.1338
Largest FMI = 0.2486
DF adjustment: Large sample DF: min = 400.43
avg = 3,078.56
max = 20,956.31
Model F test: Equal FMI F( 26,43237.9) = 39.86
Within VCE type: OIM Prob > F = 0.0000

-------------------------------------------------------------------------------
milch | Coef. Std. Err. t P>|t| [95% Conf. Interval]
--------------+----------------------------------------------------------------
comhigh | .1425119 .0721656 1.97 0.048 .0010097 .2840142
sexmil | .6263848 .0661955 9.46 0.000 .4965888 .7561807
tbi | .4585366 .0973029 4.71 0.000 .2678153 .6492578
ptsd | 1.010301 .0813186 12.42 0.000 .8509028 1.1697
|
racenew2 |
black | -.3218594 .0799783 -4.02 0.000 -.4787874 -.1649314
latino | -.163462 .073251 -2.23 0.026 -.3071686 -.0197554
other | -.0083174 .0753466 -0.11 0.912 -.1560773 .1394426
|
age | -.0427948 .0346692 -1.23 0.217 -.1108235 .025234
sex | .0494975 .0573249 0.86 0.388 -.0629349 .1619299
|
mar2 |
nev mar | -.1735742 .0998987 -1.74 0.082 -.369509 .0223605
div | .0958684 .1033094 0.93 0.353 -.1066996 .2984364
wid | .8181701 .3672285 2.23 0.026 .0980218 1.538318
|
cohab3 |
nocohab | .286816 .0806224 3.56 0.000 .1286905 .4449414
single | .2069275 .1004776 2.06 0.040 .0099337 .4039213
|
cursep |
yes | .7304501 .1092601 6.69 0.000 .5161606 .9447396
child | -.019906 .027181 -0.73 0.464 -.0732026 .0333906
edu | .0892316 .0471056 1.89 0.058 -.0031948 .1816579
rank | -.1171391 .0217602 -5.38 0.000 -.1598197 -.0744585
|
serv2 |
Army | .588866 .0722046 8.16 0.000 .4473109 .730421
Navy | .2223117 .0762556 2.92 0.004 .0727974 .371826
Marine Corps | .4289124 .0729306 5.88 0.000 .2858345 .5719903
Coast Guard | .0760331 .0920392 0.83 0.409 -.1045108 .256577
|
deploy |
no | .0990251 .0567606 1.74 0.081 -.012271 .2103211
relat | .1254324 .0174761 7.18 0.000 .0911451 .1597196
relsal | -.0007786 .046824 -0.02 0.987 -.0928302 .091273
prayst | .236341 .0294671 8.02 0.000 .1785513 .2941306
_cons | -3.863199 .1079569 -35.78 0.000 -4.075048 -3.651349
-------------------------------------------------------------------------------


mimrgns racenew2, at (ptsd=1)

Multiple-imputation estimates Imputations = 25
Predictive margins Number of obs = 39,877
Average RVI = 0.1207
Largest FMI = 0.0842
DF adjustment: Large sample DF: min = 3,428.26
avg = 7,253.81
Within VCE type: Delta-method max = 16,438.82

Expression : Linear prediction (log odds), predict(xb)
at : ptsd = 1

------------------------------------------------------------------------------
| Margin Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
racenew2 |
white | -2.015443 .0812863 -24.79 0.000 -2.174773 -1.856113
black | -2.337302 .1068825 -21.87 0.000 -2.546852 -2.127752
latino | -2.178905 .1009347 -21.59 0.000 -2.376803 -1.981006
other | -2.02376 .1019156 -19.86 0.000 -2.223558 -1.823962
------------------------------------------------------------------------------

[pgr451]

Sorry, here are the commands and the mimrgns output. Again, am I missing something? I thought predicted probabilities cannot exceed 1?


mi estimate, esampvaryok post cmdok: logit milch comhigh sexmil tbi ptsd i.racenew2 age sex i.mar2 i.cohab3 i.cursep child edu rank i.serv2 i.deploy relat relsal prayst


mimrgns racenew2, at (ptsd=1)

Multiple-imputation estimates Imputations = 25
Predictive margins Number of obs = 39,877
Average RVI = 0.1207
Largest FMI = 0.0842
DF adjustment: Large sample DF: min = 3,428.26
avg = 7,253.81
Within VCE type: Delta-method max = 16,438.82

Expression : Linear prediction (log odds), predict(xb)
at : ptsd = 1

------------------------------------------------------------------------------
| Margin Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
racenew2 |
white | -2.015443 .0812863 -24.79 0.000 -2.174773 -1.856113
black | -2.337302 .1068825 -21.87 0.000 -2.546852 -2.127752
latino | -2.178905 .1009347 -21.59 0.000 -2.376803 -1.981006
other | -2.02376 .1019156 -19.86 0.000 -2.223558 -1.823962
------------------------------------------------------------------------------

[Nick Cox]

pgr451: Your profile shows a name, Sean Vina. I'd email the administrators to ask them to use that as your identifier to match our requested preference for full real names. Use the CONTACT US button at bottom right.

[daniel klein]

First, note that you specify lots of options with your mi estimate call that are dangerous and partly unnecessary. You neither need the post option, from what you show here, nor do you need the cmdok option with a logit model as this is supported by mi estimate anyway. Also, make sure you understand what esampvaryok does, why you (think you) need it and whether the serious implications that it raises are a problem for your research.

Concerning your question, please do read the help file for mimrgns carefully. You will find that mimrgns, by default reports the linear prediction, not predicted probabilities, which is also what the output tells you.

Nick already mentioned the preference for full real names. Please avoid asking the same question in different threads. Thank you.

Best
Daniel

[pgr451]

Sorry about the name. I sent a request to make the change. Is there a way to code it so I get Predicted probability rather than linear?

-Sean Vina

[daniel klein]

I do not want to seem rude, but I refuse to answer that question here. In your own best interest, do read the help file, carefully.

Best
Daniel

[Sebby Zajac]

I just wanted to ask... is there no way to have mimrgns provide reliable predicted means?

In my unadjusted complete case analysis, margins outputs average marginal effects between groups as predicted mean differences with CIs, and with cost data this is very useful to know, eg:

. glm directc i.group,  family(gamma) link(log)

Iteration 0:   log likelihood = -6824.2562  
Iteration 1:   log likelihood = -6776.8721  
Iteration 2:   log likelihood = -6776.7452  
Iteration 3:   log likelihood = -6776.7452  

Generalized linear models                         No. of obs      =        785
Optimization     : ML                             Residual df     =        781
                                                  Scale parameter =    1.48184
Deviance         =  885.5446575                   (1/df) Deviance =    1.13386
Pearson          =  1157.316723                   (1/df) Pearson  =    1.48184

Variance function: V(u) = u^2                     [Gamma]
Link function    : g(u) = ln(u)                   [Log]

                                                  AIC             =   17.27578
Log likelihood   = -6776.745219                   BIC             =  -4320.354

------------------------------------------------------------------------------
             |                 OIM
 directcosts |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       group |
     group2  |  -.1954158   .1219055    -1.60   0.109    -.4343461    .0435145
     group3  |   .0742704   .1492613     0.50   0.619    -.2182763    .3668172
     group4  |  -.1667719    .137437    -1.21   0.225    -.4361435    .1025997
             |
       _cons |   7.743366   .1025361    75.52   0.000     7.542399    7.944333
------------------------------------------------------------------------------

. margins i.group, pwcompare

Pairwise comparisons of adjusted predictions
Model VCE    : OIM

Expression   : Predicted mean directcosts, predict()

-------------------------------------------------------------------
                  |            Delta-method         Unadjusted
                  |   Contrast   Std. Err.     [95% Conf. Interval]
------------------+------------------------------------------------
            group |
group2 vs group1  |  -409.3716   267.5076     -933.6769    114.9336
group3 vs group1  |   177.8053   358.4899     -524.8219    880.4325
group4 vs group1  |  -354.2527   296.3612     -935.1101    226.6046
group3 vs group2  |   587.1769    297.049      4.971489    1169.382
group4 vs group2  |   55.11886   218.0668     -372.2842    482.5219
group4 vs group3  |  -532.0581   323.2767     -1165.669    101.5525
-------------------------------------------------------------------

BUT, some of my cost data is imputed, and with mimrgns, the linear predictions are much less useful in describing the mean cost differences between groups...

. mi estimate: glm directc i.group,  family(gamma) link(log)

Multiple-imputation estimates                   Imputations       =         20
Generalized linear models                       Number of obs     =        912
                                                Average RVI       =     0.0274
                                                Largest FMI       =     0.0360
DF adjustment:   Large sample                   DF:     min       =  14,797.08
                                                        avg       =  46,722.41
                                                        max       = 119,453.15
Model F test:       Equal FMI                   F(   3,60830.0)   =       1.15
Within VCE type:          OIM                   Prob > F          =     0.3280

------------------------------------------------------------------------------
 directcosts |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       group |
     group2  |  -.1471588   .1126986    -1.31   0.192     -.368062    .0737443
     group3  |   .0309784   .1385526     0.22   0.823    -.2405824    .3025393
     group4  |  -.1265174   .1275897    -0.99   0.321    -.3766037    .1235689
             |
       _cons |   7.753727   .0956468    81.07   0.000     7.566256    7.941198
------------------------------------------------------------------------------

. mimrgns i.group, pwcompare

Multiple-imputation estimates                   Imputations       =         20
Pairwise comparisons of adjusted predictions    Number of obs     =        912
                                                Average RVI       =     0.0274
                                                Largest FMI       =     0.0543
DF adjustment:   Large sample                   DF:     min       =   6,506.47
                                                        avg       = 171,972.24
Within VCE type: Delta-method                           max       = 608,469.89

Expression   : Linear prediction, predict(xb)

-------------------------------------------------------------------
                  |   Contrast   Std. Err.     [95% Conf. Interval]
------------------+------------------------------------------------
            group |
group2 vs group1  |  -.1471588   .1126986      -.368062    .0737443
group3 vs group1  |   .0309784   .1385526     -.2405824    .3025393
group4 vs group1  |  -.1265174   .1275897     -.3766037    .1235689
group3 vs group2  |   .1781373   .1168412     -.0508747    .4071492
group4 vs group2  |   .0206414   .1031085     -.1814718    .2227547
group4 vs group3  |  -.1574958   .1304952     -.4132629    .0982712
-------------------------------------------------------------------

What are my options? With skewed cost data, comparing the means is difficult so I opted for a glm model with margins to describe average marginal effects...

I've tried bootstrapping mean differences with confidence intervals, and bootstrap hypothesis testing (e.g. example #3 in Stata's bs help manual), but they don't seem to play well with MI data sets...

I appreciate any and all help.

[daniel klein]

Even though it is not perfectly clear from your question, I understand there is no technical difficulty in explicitly stating the prediction you wish.

mimrgns i.group , pwcompare predict(mu)

will combine the predicted mean differences. The substantial question is whether these are approximately normal so Rubin's rules apply. In a sufficiently large sample I would be optimistic.

In general, I do not fully get the idea of fitting a generalized model then approximate the results of a linear one. If you want that, just go with the linear model in the first place. Maarten Buis has made this point several times on the list.

Best
Daniel

[Sebby Zajac]

Even though it is not perfectly clear from your question, I understand there is no technical difficulty in explicitly stating the prediction you wish.

mimrgns i.group , pwcompare predict(mu)

will combine the predicted mean differences. The substantial question is whether these are approximately normal so Rubin's rules apply. In a sufficiently large sample I would be optimistic.

In general, I do not fully get the idea of fitting a generalized model then approximate the results of a linear one. If you want that, just go with the linear model in the first place. Maarten Buis has made this point several times on the list.

Best
Daniel

You'll have to forgive my ignorance (this is just a school project), but as far as I've read, the reason for GLMs is the ability to specify mean and variance, so it avoids distributional and re-transformation issues with cost analysis (with the use of recycled predictions)?

[daniel klein]

My comment was more targeted at using average marginal effects than on specifying an appropriate model for your data. The former are usually applied to approximate the parameters of a linear model from the results obtained from a non-linear (or generalized) one. But if we want (to interpret) a linear (additive) effect, then why not fitting the respective model that gives us this effect in the first place? Why go through a model from which we apparently do not really understand the results? The common argument for using average marginal effects is facilitation of the interpretation and understanding of a somehow complicated underlying model. But this claim is not really true, because we are essentially no longer interpreting the results of the appropriate mode, but an approximation of a simpler linear one. The point is that if there is a good reason to use something better suited than a simple linear model, then I think we need to interpret the results of this model in terms of the parameters estimated by this model.

Best
Daniel