metaDTA AUC(95%CI) numerical value output

[Javier Arredondo Montero]

Good morning,

I have a question I hope someone here might be able to help me with.

When I run a MIDAS model (midas tp fp fn tn, res(all)), I obtain the summary output without issues, including the numerical AUC and its 95% confidence interval. However, the reported AUC value seems artificially high, and I suspect this may be due to insufficient handling of threshold effects—any insights on this would be greatly appreciated.

In contrast, I have been unable to obtain a numerical AUC and corresponding 95% CI using either metadta or metandi. Is this possible? Could anyone kindly share the appropriate code if so?

I’ve attached the output from all three models in case it’s helpful. There seems to be clear evidence of a negative slope—suggestive of a threshold effect—across all three approaches. The estimated correlation between logit-sensitivity and logit-specificity consistently hovers around –0.36 (midas, metandi, and metadta). Moreover, between-study heterogeneity is substantial: metadta reports tau² values of 1.06 for sensitivity and 1.48 for specificity.

Thank you very much in advance.

Best regards,
[Your Name]


////////////////////////////////////////////

midas tp fp fn tn, res(all)

SUMMARY DATA AND PERFORMANCE ESTIMATES


Number of studies = 26

Reference-positive Units = 1128

Reference-negative Units = 1365

Pretest Prob of Disease = 0.45


Deviance = 274.5

AIC = 284.5

BIC = 294.2


BICdiff = 147.9


Correlation (Mixed Model)= -0.36

Proportion of heterogeneity likely due to threshold effect = 0.13


Interstudy variation in Sensitivity: ICC_SEN = 0.24, 95% CI = [ 0.10- 0.39]



Interstudy variation in Sensitivity: MED_SEN = 0.73, 95% CI = [ 0.66- 0.81]


Interstudy variation in Specificity: ICC_SPE = 0.31, 95% CI = [ 0.14- 0.48]



Interstudy variation in Specificity: MED_SPE = 0.76, 95% CI = [ 0.69- 0.85]



ROC Area, AUROC = 0.96 [0.94 - 0.97]


Heterogeneity (Chi-square): LRT_Q = 96.126, df =2.00, LRT_p =0.000

Inconsistency (I-square): LRT_I2 = 98, 95% CI = [ 97- 99]



Parameter Estimate 95% CI

Sensitivity 0.86 [ 0.79, 0.91]

Specificity 0.94 [ 0.90, 0.96]

Positive Likelihood Ratio 14.1 [ 8.3, 24.0]

Negative Likelihood Ratio 0.15 [ 0.10, 0.22]

Diagnostic Odds Ratio 95 [ 50, 181]


metandi tp fp fn tn

Refining starting values:

Iteration 0: Log likelihood = -140.34194
Iteration 1: Log likelihood = -139.08738
Iteration 2: Log likelihood = -137.28564
Iteration 3: Log likelihood = -137.25924

Performing gradient-based optimization:

Iteration 0: Log likelihood = -137.25924
Iteration 1: Log likelihood = -137.25864
Iteration 2: Log likelihood = -137.25864

Meta-analysis of diagnostic accuracy

Log likelihood = -137.25864 Number of studies = 26
------------------------------------------------------------------------------
| Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
Bivariate |
E(logitSe) | 1.817829 .2386115 1.350159 2.285499
E(logitSp) | 2.735782 .2929618 2.161588 3.309977
Var(logitSe) | 1.060434 .4155142 .4919871 2.285669
Var(logitSp) | 1.474726 .5893011 .6738621 3.227393
Corr(logits) | -.3571722 .2664412 -.7496772 .2212176
-------------+----------------------------------------------------------------
HSROC |
Lambda | 4.493327 .3317465 3.843116 5.143538
Theta | -.2726054 .3257408 -.9110456 .3658348
beta | .1648972 .2706494 0.61 0.542 -.3655658 .6953602
s2alpha | 1.607763 .809599 .5992313 4.313699
s2theta | .8485989 .2971891 .4271711 1.685788
-------------+----------------------------------------------------------------
Summary pt. |
Se | .8603054 .0286763 .7941556 .9076689
Sp | .9391053 .0167535 .8967466 .9647695
DOR | 94.97475 31.23563 49.84964 180.9482
LR+ | 14.12777 3.797831 8.341673 23.9273
LR- | .1487528 .0299895 .1001977 .2208376
1/LR- | 6.72256 1.355311 4.528215 9.980272
------------------------------------------------------------------------------
Covariance between estimates of E(logitSe) & E(logitSp) -.0172989



. metadta tp fp fn tn, studyid(id)

*********************************** Fitted model ***************************************
tp ~ binomial(se, tp + fn)
tn ~ binomial(sp, tn + fp)
logit(se) = mu_lse + id_lse
logit(sp) = mu_lsp + id_lsp
id_lse, id_lsp ~ biv.normal(0, sigma)
Number of observations = 26
Number of studies = 26

************************************************** **************************************

************************************************** **************************************

Between-study heterogeneity statistics
covar rho
-0.45 -0.36
Tau.sq I^2(%)
Generalized 1.37 61.22
Sensitivity 1.06 69.91
Specificity 1.48 55.10

LR Test: RE vs FE model
Chi2 degrees of
statistic freedom p-val
159.77 3 0.0000
************************************************** **************************************
Study specific test accuracy: Absolute Measures
************************************************** **************************************

| Sensitivity | Specificity
Study | Estimate [95% Conf. Interval] | Estimate [95% Conf. Interval]
--------------------------------------------------------------------------------------+--------------------------------------------+--------------------------------------------
Quillin et al. (1992)(Color Doppler) | 1.00 0.69 1.00 | 1.00 0.85 1.00
Quillin et al. (1994)(Color Doppler) | 0.87 0.73 0.96 | 0.97 0.89 1.00
Patriquin et al. (1996)(Color Doppler) | 1.00 0.75 1.00 | 1.00 0.69 1.00
Lim et al. (1996) (retrospective)(Color/Duplex Doppler) | 0.88 0.76 0.95 | 1.00 0.83 1.00
Lim et al. (1996) (prospective)(Color/Duplex Doppler) | 1.00 0.69 1.00 | 1.00 0.79 1.00
Pinto et al. (1998) (Color Doppler) | 0.70 0.51 0.85 | 1.00 0.95 1.00
Pinto et al. (1998) (Power Doppler) | 0.93 0.78 0.99 | 1.00 0.95 1.00
Gutierrez et al. (1999) (Color Doppler) | 0.56 0.31 0.78 | 0.98 0.93 1.00
Kessler et al. (2003)(Color Doppler) | 0.52 0.38 0.66 | 0.96 0.86 1.00
Incesu et al. (2004)(Power Doppler) | 0.74 0.57 0.88 | 0.93 0.68 1.00
Incesu et al. (2004)(Contrast-Enhanced Power Doppler) | 1.00 0.90 1.00 | 0.93 0.68 1.00
Baldisserotto et al. (2006)(Color Doppler)(If any px considered diagnostic for AA) | 0.92 0.73 0.99 | 0.61 0.39 0.80
Gaitini et al. (2007)(Color Doppler) | 0.74 0.64 0.83 | 0.97 0.95 0.99
Xu et al. (2016)(Color Doppler)(Type 2 flow considered diagnostic for AA) | 0.57 0.39 0.74 | 0.95 0.86 0.99
Uzunosmanoglu et al. (2017)(Color Doppler) | 0.93 0.82 0.99 | 0.86 0.57 0.98
Shin et al. (2017)(Spectral Doppler)(PSV) | 0.89 0.74 0.97 | 0.95 0.85 0.99
Shin et al. (2017)(Spectral Doppler)(RI) | 0.64 0.46 0.79 | 0.96 0.88 1.00
Aydin et al. (2019)(Color Doppler/Power Doppler) | 0.63 0.55 0.71 | 0.91 0.85 0.96
Bakhshandeh et al. (2022)(Spectral Doppler)(PSV) | 0.95 0.88 0.98 | 0.95 0.85 0.99
Bakhshandeh et al. (2022)(Spectral Doppler)(RI) | 0.91 0.83 0.96 | 0.86 0.74 0.94
El-Aleem et al. (2024)(Spectral Doppler)(PSV) | 0.98 0.91 1.00 | 0.75 0.53 0.90
El-Aleem et al. (2024)(Spectral Doppler)(RI) | 0.83 0.71 0.92 | 0.79 0.58 0.93
Saini et al. (2024)(Spectral Doppler)(PSV) | 0.94 0.73 1.00 | 0.55 0.32 0.76
Saini et al. (2024)(Spectral Doppler)(RI) | 0.83 0.59 0.96 | 0.59 0.36 0.79
Anuj et al.(2025)(Spectral Doppler)(PSV) | 0.84 0.64 0.95 | 0.92 0.78 0.98
Anuj et al.(2025)(Spectral Doppler)(RI) | 0.64 0.43 0.82 | 0.94 0.81 0.99
Overall | 0.86 0.79 0.91 | 0.94 0.90 0.96
--------------------------------------------------------------------------------------+--------------------------------------------+--------------------------------------------
| |

.

[Tiago Pereira]

Hi, Javier.

It is very challenging to provide you with some tips/suggestions without additional information on the PIRO question.

[Javier Arredondo Montero]

Thank you very much for your response, Tiaro.
This is a DTA meta-analysis on the diagnostic performance of an ultrasound submodality (Doppler) for the diagnosis of acute appendicitis.
However, my question was more focused on the technical aspect of metaDTA: I would like to know which Stata modules are available to calculate a pooled AUC (numerical values with 95% CI) using bivariate models, apart from midas, and what the correct syntax would be for that.
Thank you again.

[Tiago Pereira]

What type of data do you have? In other words, do you have a 2x2 table for all studies?

[Javier Arredondo Montero]

Yes, I have the usual contingency table with the ID of each study and its TP, FP, TN, and FN.

[Tiago Pereira]

I believe answers to your questions can be found here:

1) https://onlinelibrary.wiley.com/doi/...1002/jrsm.1634
2) https://link.springer.com/article/10...90-021-00747-5
3) https://journals.sagepub.com/doi/pdf...867X0900900203

Since there is no description on how the test is performed, I am unable to conclude on the threshold effects. If the imaging technique is a standard doppler, it is unclear to me how threshold effects would operate.

[Javier Arredondo Montero]

Dear Tiago,

Thank you very much for the resources you provided—your help has been extremely valuable.

In this scenario, most authors applied a dichotomous assessment of the index test (e.g., presence vs. absence of blood flow in the appendix). However, some studies used a polytomous scale (e.g., 0 = no flow, 1–2 = sparse-to-moderate flow, 3–4 = substantial flow), while others employed continuous variables such as PSV or RI, often with a defined cut-off (in some cases determined post hoc).

The model I shared here in the forum corresponds to the overall analysis including all Doppler modalities, which understandably shows higher heterogeneity due to the inclusion of clinically diverse subgroups. For that reason, I subsequently performed subgroup analyses by modality (e.g., color Doppler, spectral Doppler), as these differ substantially in their technical characteristics and diagnostic profiles.

To me, the pooled estimates of the present model indicate substantial between-study heterogeneity, particularly in sensitivity (τ² = 1.06; I² = 69.9%) and specificity (τ² = 1.48; I² = 55.1%), with generalized heterogeneity values of τ² = 1.37 and I² = 61.2%. Moreover, the observed negative correlation between sensitivity and specificity (ρ = –0.36; covariance = –0.45) seems to indicate a threshold effect.

Any additional input is of course welcome.

Once again, thank you very much for your support.

Kind regards,

[Tiago Pereira]

Hard to tell if there is important heterogeneity based on I², which is not often not very useful when the studies are large. It is also possible that your estimates are inaccurate due to the shared reference test in some studies.