Using mi data in two-stage IPD-MA

[Kristina Vingrys]

Hi all,

Does anyone have any experience using multiple imputation with a two-stage IPD-MA model?

I am having difficulty incorporating the the *mi estimate* command with ipdmetan syntax so that it analyses the pooled mi data, rather than the multiple copies. In my ipdmetan syntax, I understand that I need to use the mi estimate prefix to specify use of mi data. Whilst *mi estimate* works with ': estimation command . . .' (regress) this results in analysis of the multiplied data points rather than the pooled mi data. When I add 'mi estimate: ipdmetan' to the first part of the syntax I get an error message (subcommand is unrecognized).

Thank you in advance for any suggestions.

Kind regards,
Kristina

[David Fisher]

Hi Kristina,

Apologies for the late response, but I have now made available an update (via SSC) to ipdmetan, in which the prefix mi estimate, post should now work. I'd be interested to know what you think, as I also have limited experience in combining meta-analysis with MI.

Thanks,

David.

[Robin Fatch]

Hi David,

I am trying to use ipdmetan with multiply imputed data, and am having some trouble with my code.

In the observed data, my code looks something like this:

ipdmetan, study(study):  glm outcome var1, family(binomial) link(logit) 

But how do I incorporate the prefix mi estimate, post that you mention above, for the MI data? I think I must be specifying things slightly wrong... I've tried some different variations with no luck. I must not be using the "post" option correctly as I'm getting some error messages (a few examples below)...

mi estimate, post ipdmetan, study(study):  glm outcome var1, family(binomial) link(logit)
*error received: invalid 'study'

mi estimate, post: ipdmetan, study(study):  glm outcome var1, family(binomial) link(logit)
*error received: command prefix ipdmetan: not allowed 

Any ideas/suggestions?

Thanks!
Robin

[David Fisher]

Hi Robin Fatch ,

At present, ipdmetan with multiply imputed data works simply by (a) first, carrying out the imputation within each study separately; and then (b) performing a standard meta-analysis of the results. That is, the MI is not part of the (second stage of) meta-analysis, but is instead part of the analysis of each individual trial prior to meta-analysis (i.e. the first stage).
Therefore, the ipdmetan command needs to be specified first, followed by a colon, followed by mi estimate, post : <command>.

Here is an example using a Stata-supplied dataset:

use https://www.stata-press.com/data/r18/mheart5, clear
count
mi set mlong
mi register imputed age bmi
gen byte study = mod(_mi_id, 3)
mi impute mvn age bmi = attack smokes hsgrad female, add(10)

* Unimputed
glm attack smokes age bmi hsgrad female i.study if _mi_m==0, family(binomial) link(logit)
ipdmetan, study(study) nograph : glm attack smokes age bmi hsgrad female if _mi_m==0, family(binomial) link(logit)

* Imputed
mi estimate: glm attack smokes age bmi hsgrad female i.study, family(binomial) link(logit)
ipdmetan, study(study) nograph : mi estimate, post: glm attack smokes age bmi hsgrad female, family(binomial) link(logit)

Best wishes,
David.

[Robin Fatch]

Hi David!

Thanks for your reply. So, this sounds exactly like what I need. I carried out the imputation within each study, combined those imputed datasets, and now want to analyze those data.

Anyways, I tried out the example code that you shared, with the same Stata-supplied dataset. It all seemed to work fine up until the last line of code, which did not work for me...

ipdmetan, study(study) nograph : mi estimate, post: glm attack smokes age bmi hsgrad female, family(binomial) link(logit)

I received the following message:
mi may not be used in this context
r(199);

Any thoughts?

Thanks,
Robin

(I am using Stata 18.0)

[David Fisher]

Hi Robin,
I am a little mystified, as the code above runs perfectly on my machine (also Stata 18.0).
Could you check whether you have ipdmetan v4.03 installed? Please type into Stata: which ipdmetan
Thanks,
David.

[Robin Fatch]

Ok! I think you are on to something here. Originally I had version 1.06 installed (!). Which is probably quite old compared to 4.03.

So, now I have v4.03 installed

. which ipdmetan
c:\ado\plus\i\ipdmetan.ado
*! version 4.03  David Fisher  12oct2022 

Then, when trying ipdmetan in the non-MI (i.e. observed) sample data this time, I got an error about my version of metan. So I installed/updated metan as well, to v4.07.

. which metan
c:\ado\plus\m\metan.ado
*! version 4.07  15sep2023
*! Current version by David Fisher
*! Previous versions by Ross Harris and Michael Bradburn 

And now, ipdmetan works with mi estimate! (I'm assuming my output below from the sample data matches what is expected) Awesome!!!

. ipdmetan, study(study) nograph : mi estimate, post: glm attack smokes age bmi hsgrad female, family(binomi
> al) link(logit)

Caution: prefix command supplied to ipdmetan. Please check estimates carefully

Studies included: 3
Participants included: 154

Meta-analysis pooling of main (treatment) effect estimate attack:smokes
using the common-effect inverse-variance model

--------------------------------------------------------------------
study                |   Effect    [95% Conf. Interval]   % Weight
---------------------+----------------------------------------------
0                    |     1.216     -0.131     2.562      32.97
1                    |     0.436     -0.746     1.619      42.78
2                    |     2.130      0.560     3.701      24.25
---------------------+----------------------------------------------
Overall, IV          |     1.104      0.331     1.877     100.00
--------------------------------------------------------------------

Test of overall effect = 0:  z =   2.799  p = 0.005


Heterogeneity measures, calculated from the data
with Conf. Intervals based on non-central chi² (common-effect) distribution for Q
---------------------------------------------------------
Measure              |     Value      df      p-value
---------------------+-----------------------------------
Cochran's Q          |      2.89        2      0.235
                     |            -[95% Conf. Interval]-
H                    |     1.203     1.000     2.256
I² (%)               |     30.9%      0.0%     80.4%
---------------------------------------------------------
H = relative excess in Cochran's Q over its degrees-of-freedom
I² = proportion of total variation in effect estimate due to between-study heterogeneity (based on Q) 

Thank you so very much!!!
Robin

[David Fisher]

Hi Robin,
Yes I can confirm that the results I get are sufficiently similar.
(Note that MI is a random process, so we'd only get exactly the same results if we both pre-set our randomization seed -- I probably should have thought of that and added it to my code!)
Best wishes,
David.

[Robin Fatch]

Hi again David Fisher,

So I am continuing work on our analyses with MI data, using ipdmetan. (thanks again for your earlier help) I have a few new questions about ipdmetan:

- is it possible to create a funnel plot within ipdmetan? (maybe using "confunnel" somehow?)
- is there any type of continuity correction that I can add to allow for studies with 0 events? (e.g. "cc(0.5)"?)

I don't see anything in the help file about these, so I'm guessing they are not available, but just wanted to double check...

Thanks!!
Robin

[David Fisher]

Hi Robin,

If you use the saving() or clear options to ipdmetan, you will obtain a dataset similar to a standard summary-data meta-analysis dataset. Then, you can use confunnel etc. to perform any additional tests, plots etc. relevant to meta-analysis. This is a result of ipdmetan implementing two-stage IPD meta-analysis, where the second stage is almost exactly similar to standard summary-data meta-analysis. Note that you may have to prepare this dataset very slightly, e.g. limiting to observations where the variable _USE takes on value = 1 (or potentially also 2), to strip away extra observations used in creation of the forest plot. This information is all given within the documentation for metan.

For your second question, I'm a little confused. Continuity correction is typically used at the summary level, where (e.g.) 0.5 is added to the total count of observations within a particular category. I'm not sure how this applies to IPD? If you have problems with fitting, e.g. perfect-prediction of a binary outcome, you will need to resolve this using an appropriate statistical model (e.g. penalized likelihood).

BW,
David.

[Robin Fatch]

Thanks David! This was very helpful.

You're right - we were having perfect prediction issues for a very small study, so it was being dropped.

Thanks,
Robin

[Sifan Cao]

Hi Robin Fatch ,

At present, ipdmetan with multiply imputed data works simply by (a) first, carrying out the imputation within each study separately; and then (b) performing a standard meta-analysis of the results. That is, the MI is not part of the (second stage of) meta-analysis, but is instead part of the analysis of each individual trial prior to meta-analysis (i.e. the first stage).
Therefore, the ipdmetan command needs to be specified first, followed by a colon, followed by mi estimate, post : <command>.

Here is an example using a Stata-supplied dataset:

use https://www.stata-press.com/data/r18/mheart5, clear
count
mi set mlong
mi register imputed age bmi
gen byte study = mod(_mi_id, 3)
mi impute mvn age bmi = attack smokes hsgrad female, add(10)

* Unimputed
glm attack smokes age bmi hsgrad female i.study if _mi_m==0, family(binomial) link(logit)
ipdmetan, study(study) nograph : glm attack smokes age bmi hsgrad female if _mi_m==0, family(binomial) link(logit)

* Imputed
mi estimate: glm attack smokes age bmi hsgrad female i.study, family(binomial) link(logit)
ipdmetan, study(study) nograph : mi estimate, post: glm attack smokes age bmi hsgrad female, family(binomial) link(logit)

Best wishes,
David.

Hi David,

I've followed your example code here and wrote my code like this:

mi estimate, eform noisily cmdok: meglm Livebirth ib1.Group Age, nocons || Study:, family(poisson) link(log) nolog eform vce(cluster Study)
ipdmetan, study(Study): mi estimate, eform noisily cmdok: meglm Livebirth ib1.Group Age, nocons || Study:, family(poisson) link(log) nolog eform vce(cluster Study)

The first line of code works fine by itself but running the ipdmetan with the same code gives me this warning:

caution: prefix command supplied to ipdmetan. Please check estimates carefully. No studies found with sufficient data to be analysed

Can you help with this?

Thanks,
Sifan

[Cristina EspinosadaSilva]

Hi David Fisher,

I’m working on a causal mediation analysis using multiply imputed individual participant data and the ipdmetan command. For the causal mediation, I’m using the inverse odds weighting approach (Tchetgen Tchetgen, Statistics in Medicine, 2013) and a user-written program to:
(1) Estimate the total effect using Poisson regression
(2) Estimate the direct effect using the same Poisson regression as above but weighted with the inverse odds weight
(3) Estimate the indirect effect by subtracting the direct effect from the total effect
(4) Exponentiate the coefficients for the total, direct, and indirect effects, and bootstrap all estimates

This user-written code worked for another analysis from a single study, but for this analysis I’m having a few issues - starting with an error after attempting to run the weighted Poisson regression model (code below).

*Poisson regression to estimate the TOTAL EFFECT (this runs fine);
ipdmetan, study(study) re(ml) nograph: mi estimate, post : poisson mi_outcome exposure

*Poisson regression weighted with inverse odds weight to estimate the DIRECT EFFECT;
ipdmetan, study(study) re(ml) nograph: mi estimate, post : poisson mi_outcome exposure [pweight=iow_mi]

*Error message: an error occurred when ipdmetan executed poisson. Try running command (including prefix commands) independently of ipdmetan to help identify the error. Error in ipdmetan.CommandLoop. r(459);

Is perhaps the issue that I need to use different syntax when referring to the weighting option? Any of your suggestions would be greatly appreciated!

Best,
Cristina