Hi,
I realise that questions on this topic are common but it is conceptually difficult.
I’m trying to compare scan derived categorical (rmra rjs) and continuous (rgps rsas) classifications of optic discs against a gold standard (roag). Some of my covariates are potential quality metrics; rsd is a marker of scan quality, while rads is a predictor of misclassification.
I’ve tried; rocreg roag rmra rjs rgps rsas , ctrlcov(rads examage gender rsd) cluster(Study) ctrlmodel(linear)
Am I using the right model or should I be analysing categorical and continuous classifications as separate groups ? Can I treat my covariates not as an adjustment but as an additional clinical tool / quality metric in this way ? And if so what’s the best way of describing my result to a clinical audience. Could I say, for instance “rads was a significant predictor of test discrimination and where a scan has an ads score below x, automated scan derived classifications should be interpreted with caution, etc..” ? Presumably I should present adjusted and unadjusted AUCs in my results ?
I've enclosed a sample dataset. In the full data set (classifier rmra, rads coef=0.94 (95% CI 0.86 – 1.01), NB in the sample dataset it’s not significant (p=0.792) for rmra.
As ever, I'd be grateful for any advice / insight
Many thanks
Ali Poostchi
Ophthalmology Registrar, Nottingham
I realise that questions on this topic are common but it is conceptually difficult.
I’m trying to compare scan derived categorical (rmra rjs) and continuous (rgps rsas) classifications of optic discs against a gold standard (roag). Some of my covariates are potential quality metrics; rsd is a marker of scan quality, while rads is a predictor of misclassification.
I’ve tried; rocreg roag rmra rjs rgps rsas , ctrlcov(rads examage gender rsd) cluster(Study) ctrlmodel(linear)
Am I using the right model or should I be analysing categorical and continuous classifications as separate groups ? Can I treat my covariates not as an adjustment but as an additional clinical tool / quality metric in this way ? And if so what’s the best way of describing my result to a clinical audience. Could I say, for instance “rads was a significant predictor of test discrimination and where a scan has an ads score below x, automated scan derived classifications should be interpreted with caution, etc..” ? Presumably I should present adjusted and unadjusted AUCs in my results ?
I've enclosed a sample dataset. In the full data set (classifier rmra, rads coef=0.94 (95% CI 0.86 – 1.01), NB in the sample dataset it’s not significant (p=0.792) for rmra.
As ever, I'd be grateful for any advice / insight
Many thanks
Ali Poostchi
Ophthalmology Registrar, Nottingham
Code:
* Example generated by -dataex-. To install: ssc install dataex clear input int StudyNumber float examage long gender float(roag rjs rmra) double(rgps rsas rads) int rsd 32 74.10815 1 1 3 . . . . . 33 77.39083 0 0 1 1 .09 .24694214986444324 .015187238254209744 33 52 85.86995 1 1 1 2 .53 .5427921697288663 .6256634527293633 26 92 68.019165 0 1 3 3 .89 .996143397435842 .1376716190840947 18 93 81.48939 0 0 1 1 .31 .15170827301718864 .9841314631499114 19 164 78.91033 1 1 . 2 .83 .9779318775180398 .8049106039784562 22 165 85.74401 1 0 3 . . . . . 175 69.98768 0 1 3 3 .57 .9601099662153239 .0654765051587221 100 176 82.00684 1 0 1 1 .9 .005071101937208 .0016991062929211471 36 205 73.42368 0 1 3 3 .72 .8232151831282817 .7009586263514439 16 206 74.25051 1 0 1 1 .1 .05154141380797728 .23165543351847362 32 214 77.23203 1 1 3 3 . .9978677806787595 .22506746778670386 47 215 79.27173 0 . . 3 .86 .2538690534890233 .9363795606599816 22 219 81.90554 1 1 3 3 .91 .9963729831576595 .2660305967250509 27 220 78.78439 1 0 3 1 .88 .8036976265174537 .5272711271103941 41 236 70.858315 1 1 3 3 .29 .5387593643840156 .7648105039215902 13 237 75.47707 1 0 1 1 .09 .8903348362387873 .06556848852313049 19 308 80.95277 1 1 1 1 .86 .3183457093452107 .7546589569171274 24 309 74.113625 0 0 3 3 .72 .3276659337258898 .3558178035810762 15 end label values gender gender label def gender 1 "F", modify label values rmra norm label values rjs norm label def norm 1 "Normal", modify label def norm 2 "Borderline", modify label def norm 3 "Abnormal", modify
