ROC analysis after multiple imputation

[Ester Olthof]

Hi everyone,


Could somebody please help me figure out how to perform ROC analyses after multiple imputation?

I’m interested in the diagnostic accuracy: sensitivity, specificity, PPV, NPV, and AUC
of: MRI, CT, and PET-CT imaging (index-tests)
compared to: pathological examination (verification test).

Unfortunately, data on pathological examination is missing in 38%, which I imputed successfully using multiple imputation by chained equations (m=20). I can calculate the diagnostic indices for all datasets separately and then, calculate the mean of these values manually according to Rubin’s rule.

However, I would like to compare the mean AUCs (M1-M20) and create a ROC-curve including the three imaging modalities. The standard STATA commands (i.e. “roccomp”, “roctab”, “rocreg”, “rocregplot”) are not working in combination with mi estimate.
Searches on the internet/manuals/"https://www.stata.com/support/faqs/statistics/combine-results-with-multiply-imputed-data/" didn’t bring me further.

Hopefully, there is someone who wants to share his/hers knowledge on this with me.


Best wishes, Ester

[Rich Goldstein]

please see #6 in https://www.statalist.org/forums/for...ng-mi-estimate

[Ester Olthof]

Dear Rich Goldstein,


Thank you very much for the recommendation, and my apologies for missing that post.
Your script is really helpful in calculating the AUC/C-statistic after imputation for my three index tests separately.

Hereby an example of MRI:
In which "mri_pb" stands for nodal status on MRI (i.e. (0) negative, (1) positive, (2) inconclusive) and "np_pb" for nodal status on pathology (i.e. (0) negative or (1) positive).

local rhs "i.mri_pb"
noi mi estimate, or saving(miest, replace): logit np_pb `rhs', vce(cluster _mi_id)
qui mi query
local M=r(M)
scalar r2=0
scalar cstat=0
qui mi xeq 1/`M': logistic np_pb `rhs'; scalar r2=r2+e(r2_p); lroc, nog; scalar cstat=cstat+r(area)
scalar r2=r2/`M'
scalar cstat=cstat/`M'
noi di "Pseudo R=squared over imputed data = " r2
noi di "C statistic over imputed data = " cstat

Additionally, I would like to compare the C-statistics for MRI / CT / and PET-CT with each other, and I would like to plot a ROC-curve.
Do you possibly have any suggestions on how to continue with that?


Kind regards, Ester

[Rich Goldstein]

not sure what you mean by "compare the C-statistics"; you can get the predicted values using -mi predict- and then use Stata's roc tools; see

h mi predict
h roc