How to evaluate perfomance of existing risk prediction model

[Chris Lenselink]

Hello,

This is my first post on this forum.

I am trying to compare the model performance of 3 logistic regression models I made in Stata to an already existing risk prediction model. I compared the logistic models' AUCs using:

stepwise, pr(.05): logistic IHD LDLreceptor SELE APN CHI3L1 EGFR RARRES2 
lroc, nograph
predict xb1, xb

stepwise, pr(.05): logistic IHD EV_IL2RA EV_TFF3 EV_CNTN1 EV_FABP4 EV_SELE EV_CXCL16 EV_IL6RA EV_TRAP EV_GP6 EV_APN EV_TNFSF13B EV_JAMA EV_tPA EV_RARRES2 EV_vWF EV_PECAM1 
lroc, nograph  
predict xb2, xb 

stepwise, pr(.05): logistic IHD APN CHI3L1 RARRES2 EV_APN
lroc, nograph
predict xb3, xb 

roccomp IHD xb1 xb2 xb3, graph summary

The already existing risk prediction model, SCORE, is presented as a variable (float) for every observation - it is a risk presented as percentage. How can I compare the predictive ability of this SCORE variable for the binary IHD variable with my 3 models?

[Clyde Schechter]

gen score = SCORE/100
roccomp IHD score xb1 xb2 xb3, graph summary

If SCORE is presented as a percentage, to use it in -roccomp-, all you need to do is convert it to a probability by dividing it by 100.

That said, if the existing risk prediction model is a reference criterion (aka "gold standard"--I dislike that term) to which all the others should be compared then you should use -rocgold- instead of -roccomp-. You should stick with -roccomp- if you regard these all as just candidate risk prediction models, which can be contrasted with each other as equally viable.