Effect size after multiple imputation

[Elisa Wallwaey]

Hi Statalisters,

I have a multiply imputed dataset (wide format, m = 20 imputation cycles) and I would like to find out about the effect size of mean differences between two groups in the sample. I know that I can calculate the effect size seperately for every imputation cycle by using mi xeq which works with the esize-command, but I would like to have an overall effect size (preferably Hedge's g) for all imputation cycles at once.
Is there a possibility to calculate this even though the esize-command does not work with mi estimate?
I just seem to be stuck on this and would very much appreciate any advice.

Best,
Elisa

[daniel klein]

I am not a big fan of effect size measures, mainly because I think they tend to be as hard, and in most cases harder, to interpret as the mean difference in its "natural" metric. I guess human beings just do not have an exactly clear idea about what a standard deviation really is or means. Anyway, I guess one could basically approach this in two ways.

Treating the effect size measure as a "final" estimate, we could merely combine the effect sizes across the imputed datasets, using Rubin's rules. Isabel Cañette and Yulia Marchenko show how to do this here. Note that effect sizes are most likely not (approximately) normal, so some kind of transformation seems desirable. This, however, still seems the easiest way. It is not clear to me whether it is appropriate, though.

Alternatively, we could try to estimate the effect size measure "by hand" using esizei. The (mi) point estimates for the group means could easily be obtained using mi estimate. I am not clear on how to get the standard deviations in both groups, though. Probably both the within and between imputation variances should be part of this calculation. It is also not clear to me, which sample size should be used, in case the group sizes differ across the imputed datasets. This second approach thus seems more complicated, but the problems I address here might not be solved by, but rather masked somehow following the first approach.

Others might have more insight.

Best
Daniel

[Elisa Wallwaey]

Thank you, Daniel.
The first suggestion worked just fine.

Best,
Elisa

[Paul Tiffin]

There is now an .ado package, miesize, I wrote. It uses Rubin's rules to calculate effect sizes and their confidence intervals from multiply imputed data. It is available to install from the SSC.