Using the metan command for regression coefficients

[Talia Sherrard]

I am looking at the association between a categorical exposure and continuous outcome in several cohorts. I have used linear regression modelling to do this, adjusting for potential covariates in each model

I would now like to meta-analyse the results from all cohorts, but am unsure of how to go about doing this. Would the metan command be appropriate? I cannot find an example of metan being used for this kind of data. Does anyone know of a more suitable option?

Many thanks for your help

[David Fisher]

Hi Talia,

To be clear: you are analysing the individual-level data from each cohort? (rather than, say, a summary estimate for each cohort taken from published articles?)

If so, what you are describing (i.e. using "metan") is commonly known as two-stage individual participant data meta-analysis (IPD-MA). If you search for this, particularly "two-stage", you should find a few examples in the literature, both practical and methodological. In particular, I have written a Stata package called ipdmetan (type ssc describe ipdmetan at the Stata command line) which automates this approach by fitting the same model (including adjustment covariates) within each cohort, pooling the results, and creating a forest plot.

The alternative approach is a one-stage model, i.e. a single "mixed" regression model fitted to all the data from all the cohorts simultaneously. There are many options here: fixed or random effects on the treatment effect and/or on the intercept; a common or cohort-specific residual error; and so on. Just make sure that cohort membership is taken into account in some way (see e.g. Abo-Zaid [1] for more on this).

Burke et al [1] and Morris et al [2] are both good articles discussing the differences between one-stage and two-stage models. Particularly for continuous (Gaussian) outcomes, the differences are negligible providing the models being compared are truly equivalent. The main downside to one-stage models is that forest plots (plus heterogeneity tests etc.) cannot be directly produced from the results (one workaround to this is to insert the one-stage result into a two-stage forest plot -- let me know if you want further guidance on this). Conversely, two-stage models are more simplistic and cannot handle more complex setups... although in your case you should be OK either way.

Finally, just be aware that if you adjust for different covariates between cohorts (e.g. if you don't have full data available for some cohorts) then strictly speaking your estimands are not consistent, and pooling them becomes questionable. This becomes more important if e.g. a certain covariate which causes a big change in the effect of interest is missing for some cohorts -- since you don't know how the effect of interest would have changed if that covariate were available. One solution here is to present both adjusted and unadjusted results; or even include "partially-adjusted" results where you only adjust for covariates which are available across all cohorts. More sophisticated methods are also possible, such as multivariate meta-analysis or imputation. All this may or may not be an issue in practice, but you should at least be aware of it.

Hope this helps. Let me know if you have any further questions, particularly about ipdmetan or about forest plots.

Thanks,

David.


[1] Abo Zaid G et al. Individual participant data meta-analyses should not ignore clustering. Journal of Clinical Epidemiology 2013; 66: 865-873

[2] Burke DL, Ensor J, Riley RD. Meta-analysis using individual participant data: one-stage and two-stage approaches, and why they may differ. Statistics in Medicine 2017; 36: 855-875

[3] Morris TP, Fisher DJ, Kenward MG, Carpenter JR. Meta-analysis of Gaussian individual patient data: two-stage or not two-stage? Statistics in Medicine 2018; 37: 1419-1438.

[Talia Sherrard]

Hi David,

Thank you very much for your informative answer! Yes I am using individual-level data from several cohorts. They have significant differences in location, study design, confounding structure etc so I think from your answer a two-stage model would be more suitable. The ipdmetan package looks like a good way of doing this.

I am currently still grappling with basic metan as I am getting odd output. For some reason metan coefficient standard_error, fixed and metan coefficient standard_error, random are generating identical results with no change in weighting. My I^2, H^2 and tau^2 are also bizarre. Do you have any idea what may be causing the issue?

	Study	ES	[95% Conf.	Interval]	% Weight
1		0.003	-0.016	0.023	57.12
2		0.009	-0.014	0.031	42.88
D+L	pooled	ES	0.006	-0.009	0.021	100.00

Heterogeneity chi-squared = 0.11 (d.f. = 1) p = 0.741
I-squared (variation in ES attributable to heterogeneity) = 0.0%
Estimate of between-study variance Tau-squared = 0.0000


If you were able to shed any light on what may be the problem that would be really appreciated!

Thanks,
Talia

[David Fisher]

Hi Talia,

It's simply that the two results are so similar (strictly speaking: the point estimate of the difference is small relative to its standard error) that heterogeneity is negligible, and hence I-squared and tau-squared are zero, or very close to zero. Since (standard) random-effects weights are the same as fixed-effects weights except for adding tau-squared in the denominator, if tau-squared is zero then the weights will not change.

Be aware that heterogeneity is not easy to estimate, especially when there are few studies (as here). In this example, I think the estimate of tau-squared as zero is OK: the two studies are very similar. But in other cases, it might not be so clear-cut. Various random-effects models are available, with different properties, and which work well in different scenarios. The default DerSimonian-Laird method uses a simple, non-iterative formula, but often gives a zero result if heterogeneity is small (as you found). Some popular alternatives include REML, profile-likelihood and Hartung-Knapp-Sidik-Jonkman (HKSJ), all of which are implemented in admetan (the "metan" equivalent within the "ipdmetan" package). There is no perfect model, unfortunately ... but as in all statistics, the more observations (in this context, "studies") the better!

Hope that helps,

David.