meta-analysis of paired data

[Marios Politis]

Hi all,

I am conducting a meta-analysis of prevalence of a vaccine side effect (let's say fever) within a specific group of people. The data I have collected include prevalence of fever after different vaccinations of the same vaccine (1st dose, 2nd dose etc). Of course, those data are paired, but I treated them as independent and included all the studies in one analysis because it is important for my study to measure the overall prevalence of fever after all vaccinations.

However, I conducted subgroup analysis to explore for between-study heterogeneity and as expected there was significant heterogeneity between the dose-subgroups. As Cochrane Q statistic is the test used in subgroup analysis to compare differences in effect sizes, my question is if this a proper test to be used when the data are paired.

Thanks for your help.

[Tiago Pereira]

Could you clarify the type of data you have? Are you dealing with the number of events out of a total, such as n/N?

[Marios Politis]

Thanks for your answer Tiago. Yes, n/N is the type of data.

[Tiago Pereira]

It appears you have strong evidence that prevalence differs between studies of one-dose and two-dose vaccinated individuals. The Q-test usually provides no meaningful information in meta-analysis of proportions (The test will almost always be statistically significant, even if the proportions differ only slightly). You should calculate the 95% prediction interval for the prevalence to be estimated in future studies.

If you want to investigate the association between 2 doses with higher prevalence of the event, you can add this indicator variable (1 = two doses, 0 = otherwise) and fit a random-effects meta-regression model. Since the studies are correlated, you should use -robumeta-

You need to transform your proportions to logit first and then use them in -robumeta-

 ssc install robumeta

[Bruce Weaver]

It appears you have strong evidence that prevalence differs between studies of one-dose and two-dose vaccinated individuals.

Tiago Pereira, I may have misunderstood, but I believe that Marios Politis has data for the same people following 1st and 2nd doses, not independent studies of one dose vs two doses. I.e., I think the n/N variables Marios are actually two pairs of such variables, as follows:

• n1 = # experincing fever after 1st dose
• N1 = # receiving the 1st dose
• n2 = # experincing fever after 2nd dose
• N2 = # receiving the 2nd dose

Marios, you could clear things up greatly by using -dataex- to share some data with us. If you don't want to share the actual data, "toy" data with the same structure is fine. TY.

[Tiago Pereira]

Bruce Weaver is right. We need to check the data structure.

[Marios Politis]

Thank you both for your time.

Bruce Weaver you are right, the data are paired (1st dose, 2nd dose in the same population), coming from the same individual studies rather than independent studies.
The data in main analysis:
Study | n (vaccinated people with fever) | N (vaccinated people with ANY dose)

However, in subgroup analysis, among other sources of heterogeneity (gender etc) I am exploring the effect of vaccine dose:
Study | n (people with fever after the 1st dose) | N (vaccinated people with the 1st dose) - indicator variable = 0
Study | n (people with fever after the 2nd dose) | N (vaccinated people with the 2nd dose) - indicator variable = 1

Example:

Study n N indicator (vaccine dose)
a |5 |10| 0
b |8| 20 |0
c |13 |100| 0
a |7 |10 |1
b |13 |20 |1
c |57 |100 |1

The prevalence in the different doses is expected to differ and can be explained theoretically. I am not interested in exploring this particular association. This is mostly a formal subgroup analysis to explore the observed between-study heterogeneity in the main analysis, among other sources of heterogeneity.

My main question is if the Q-test is the proper method to compare both the paired (different vaccine doses) and unpaired (different gender) proportions in subgroup analysis.

[Tiago Pereira]

You should conduct a separate meta-analysis for each dose. That's it. No comparison.

If you want to go more complex:

Let pi represent the proportion of participants after i doses (where i is either 1 or 2). To test the difference in prevalence between the number of doses, you have to calculate the difference between the correlated proportions (ES = p2−p1). Next, determine the appropriate standard error (seES), say Wald SE with a continuity correction. A meta-analysis using ES (seES) will examine the null hypothesis that p2==p1. You can also calculate a log-odds ratio and its standard error (assuming paired data).

[Bruce Weaver]

WARNING: This is still only half-baked (if that), so I hope you'll forgive me if it turns out to be a lousy idea.

Based on the data you have shown us, it looks as if all subjects had both doses of vaccine, and that you can therefore think of each study as a 2x2 table like this (using your Study 1 as an example):

I.e., if I understood, 5 of 10 Ss had fever (F+) after dose 1 in Study 1, and 7 of 10 had fever after dose 2. These are the marginal totals for the 2x2 table. If you have the information you need to fill in the cells of the table, you can compute an OR as b/c, the ratio of the counts in the two incongruent cells. I'm not completely certain, but I think that sqrt(1/b + 1/c) would give you a SE for ln(b/c). If so, you could meta-analyze the logs of the ORs.

What do you reckon?

[Tiago Pereira]

I agree with Bruce. Under assumption of large samples, that is a good approximation. In genetics, this approach is used to meta-analyze transmission/disequilibrium test studies, given the inherent paired nature of the data (similar to Mario's case).


Reference (with formulas)
Kazeem GR, Farrall M. Integrating case‐control and TDT studies. Annals of human genetics. 2005 May;69(3):329-35.

[Bruce Weaver]

Thanks for the reference, Tiago Pereira.

[Marios Politis]

Thank you for your responses.
After researching a little bit, I realized that a bivariate meta-analysis could be the proper method for this type of data, as it accounts for the within-subjects dependency. The results are almost identical to those obtained with univariate meta-analytic models using the SE of the proportions and an assumed correlation coefficient. I am not entirely sure, though, how to interpret the Q test for homogeneity. Does it compare heterogeneity between the two variables or within each variable?

[Leonardo Guizzetti]

Do you have individual level data, or only aggregate data?

[Marios Politis]

Hi Leonardo, it's aggregate data.

[Tiago Pereira]

Hi, Marios.

If you were estimating small to moderate treatment effects of interventions, then it would make a lot of sense to use a multivariate meta-analysis. Any gain in precision is welcome in that case. However, in your case, you are describing the prevalence of relatively frequent events. I hardly see any gain in precision - clinically significant - with more complex, bivariate models. Any gain in precision is unlikely to change the clinical interpretation regarding the overall prevalence. Two univariate meta-analyses would convey the same clinical interpretation concerning how frequent the events are in the populations - simpler to understand and report.

As discussed above, a meta-analysis of proportions nearly always shows statistical heterogeneity. Hence, a Q test is not informative. What is informative is the prediction interval, constructed with the random-effects estimates.

You may receive more responses if you provide real examples and outputs in Stata.