Meta-analysis of AUC values

[Bruce Weaver]

I have been asked to help with meta-analysis of AUC values. My first thought was that I could use

meta set esize se

with esize = logit(AUC). But when I started looking for info on how to compute the SE of logit(AUC), I could find any clear info on how to do it.

So, I have two questions:

1. Is using logit(AUC) as the effect size a good idea? If not, what would you advise instead?
2. How can I compute the SE of logit(AUC)?

Thanks.

[Leonardo Guizzetti]

AUC are simply proportions, so in that sense, the same machinery to meta-analyze single proportions are valid here as well. A logit transformation is a good idea (and and so is the arcsine transformation), versus the raw value. The SE is a sticky problem though. It's my experience that many authors reporting AUC neglect to report an SE or confidence interval, and there isn't a direct way to compute this (unlike the SE of a simple proportion).

Do you have access to the raw data values?

[Bruce Weaver]

Thanks for your reply, Leonardo Guizzetti. AFAIK, the folks who consulted me do not have access to the raw data for the articles.

Meanwhile, I found this BMJ article, which may help. See equations 55 and 59 on page 9 of the supplementary material. If all studies reported the AUC plus the numbers of events and non-events, then taking the square root of equation 59 should give an approximate SE for logit(AUC). What do you think?

[Leonardo Guizzetti]

Thanks for your reply, Leonardo Guizzetti. AFAIK, the folks who consulted me do not have access to the raw data for the articles.

Meanwhile, I found this BMJ article, which may help. See equations 55 and 59 on page 9 of the supplementary material. If all studies reported the AUC plus the numbers of events and non-events, then taking the square root of equation 59 should give an approximate SE for logit(AUC). What do you think?

There's some strong caveats with use of that resource, I think. First, these are defined with respect to a prediction model study in which the outcome is understood to be binary. AUC's do not need to be confined to binary endpoints. If that does apply, then equation 55 is still undefined without the variance estimate. However, the approximation they provide I think assumes a maximal variance under the null which will likely overestimate the variance in the study, and therefore might add a lot of unnecessary variability in the meta-analysis.

That said, if you're happy with those assumptions, you could proceed along that direction.

[Bruce Weaver]

Thanks again, Leonardo Guizzetti. The outcome variable for the original studies is binary, and the proposed meta-analysis of AUCs is very much a secondary analysis being done to appease a reviewer, apparently. So perhaps equation 59 is good enough under the circumstances.