After xtlogit regression estadd scalar from rocreg

[Catharina Klepsch]

Dear Stata users,

I am doing logit regressions on a pandel data set (using xtlogit). After running xtlogit I am trying to determine the forecast ability of each model using "predict, xb", and comparing the predicted outcome (obtained from "predict") with the actual outcome (dependent variable) by examining the area under the roc curve by using the Stata comand "roctab".

As I compare several different xtlogit model specifications with each other (see below model1 versus model2), I also want to compare the different predicted outcomes which each other. So I compare the different AUCs for each model with each other using "rocreg" (alternatively roccomp can be used).

My problem is now that I would like to add the p-value obtained from the command "rocreg" (H0: AUC of model 1 = AUC of model2) to my final regression table. Unfortunately that does not work and I have no idea why this is the case, as I am able to add the normal AUC value to my table.

Below an extract of my do file using Stata's union.dta:

use http://www.stata-press.com/data/r13/union.dta, clear

eststo model1: qui xtlogit union black age grade, nolog
predict xb1 if e(sample), xb
roctab union xb1 , nograph
local AUC = `r(area)'
estadd local AUC "`:di %6.3f `AUC''"


eststo model2: qui xtlogit union black age grade south, nolog
predict xb2 if e(sample), xb
roctab union xb2 , nograph
local AUC = `r(area)'
estadd local AUC "`:di %6.3f `AUC''"
rocreg union xb1 xb2, tiecorr bseed(123) nodots breps(10)
return list
estadd scalar AUCpval = r(p)

estout , stats(AUCpval AUC)


Thanks for any advice !!

[Aspen Chen]

It seems like -rocreg- started a new estimate. So your e(AUCpval) is stored separately from model2. You could force -estout- to still display the value by explicitly storing estimates from the -rocreg- line and adding the model to -estout-, but that does not sound like what you want.

Here's how I would revise it. I used -estimates- rather than -eststo- so I could restore the estimates for model 2 and add the new scalar.

est clear
use http://www.stata-press.com/data/r13/union.dta, clear

qui xtlogit union black age grade, nolog
est store m1
predict xb1 if e(sample), xb
roctab union xb1 , nograph
local AUC = `r(area)'
estadd local AUC "`:di %6.3f `AUC''"
qui xtlogit union black age grade south, nolog
est store m2
predict xb2 if e(sample), xb
roctab union xb2 , nograph
local AUC = `r(area)'
estadd local AUC "`:di %6.3f `AUC''"
rocreg union xb1 xb2, tiecorr bseed(123) nodots breps(10) // this starts a new estimate
return list
scalar pval = r(p)
est restore m2    // restore m2
estadd scalar AUCpval=pval // add the scalar

estout m1 m2, stats(AUCpval AUC)

[Catharina Klepsch]

Aspen, that's just what I needed. Just perfect so thank you very much.