Graph Observed versus Predicted

[Jean-Baptiste Lascarrou]

Dear Statalisters,

I work on prediction of outcome (neurological outcome 0 or 1) with a score which range from 0 to 200.

The discrimination and calibration of the score is good and I want to highlight the calibration on a graph Observed outcome versus Predicted and the development cohort and 3 validation cohorts.

I try:

 logit CPC_B CAHP

 margins

But without success...

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(CPC5 CAHP)
1      .
1      .
1 108.52
1      .
1 127.34
1 100.02
1  79.22
1      .
1 117.54
1 131.12
end

[Rich Goldstein]

this is the basic logic I use for such a calibration plot:

estimate model
predict newvar
lowess depvar newvar, mcolor(black%0) addplot(function y=x, range(0 1) ///
 || scatter depvar newvar, mcolor(green%20) jitter(5)) legend(off)

substitute your actual variable names; to show additional lowess plots for your "validation cohorts", you can add these into the "addplot" section of your command

note that I did not try using your data example as it does not contain the variable shown in your -logit- command

note also that there appears to be lots of missing data and you probably need to do something about that if it is representative of your actual data

note that the above has recently been studied in "Austin, PC and Steyerberg, EW (2014), "Graphical assessment of internal and external calibration of
logistic regression by using loess smoothers," _Statistics in Medicine_, 33: 517-535

[John Moran]

A more "formal" approach to this has been given by Nattino et al (SJ17-4: gr0071): "calibrationbelt -- Plot the calibration belt and compute the associated test to evaluate the calibration of binary outcome models"

john moran

[Jean-Baptiste Lascarrou]

Thanks to you Rich and John.