calculating hit rate after logistic regression

[eric chen]

hi,

I'd like to know how to calculate hit rate after a logistic regression. I have used the survey set in my data.

Also, in terms of checking the predictive power of logistic model, is there any difference between k fold cross validation and hit rate?

Thanks

[Andrew Lover]

"hit rate" is perhaps analogous to ROC analysis? see

http://www.stata.com/statalist/archi.../msg00493.html

[Rich Goldstein]

it would be helpful, to me at least, if you would define what you mean by "hit rate" - and a citation to a technical article would also be appreciated

[eric chen]

it would be helpful, to me at least, if you would define what you mean by "hit rate" - and a citation to a technical article would also be appreciated

Sorry for the ambiguity.

Here's the definition:

"The (insample) hit rate is defined as the percentage of the observations (in-sample) that is correctly predicted by the model."

Sorry I couldn't find a technical paper on this concept. But here's a relevant example:

http://www.jstor.org/stable/1392289

Thanks

[Rich Goldstein]

"estat classification" is what it appears you want - it is NOT the same thing as k-fold-cross-validation; after estimating your logistic model, just type

estat classifcation

it appears that what you want is the line marked "Correctly classified"

[eric chen]

"estat classification" is what it appears you want - it is NOT the same thing as k-fold-cross-validation; after estimating your logistic model, just type

estat classifcation

it appears that what you want is the line marked "Correctly classified"

Hi,

Thanks for the information. I have tried the command with my data but it didn't work. I think it's because I'm using a survey design. Hence the regression was

svy: logistic A B

What I'm trying to do here is to find out the predictive power of the model, i.e. whether it's a good model or not.

Apologies for my lack of technical knowledge.

Thanks

[Rich Goldstein]

first, note that "whether it's a good model or not" is not a simple consideration for any model; for logistic regression, many people look at the model using "discrimination" (area under the ROC curve) and calibration - I'm not sure how to get discrimination for a survey design (I don't use surveys much myself) but you can use "estat gof" after your command to get some information at least; note for this test that the null hypothesis is that you have a good fit so a p-value <0.05 means you reject that null; there is probably a fairly simple way to get your "hit rate" following a "svy" but I don't immediately see what it is; and, yes, "estat classification" cannot be used after "svy"; please see the FAQ for advice on how to write questions

[Andrew Lover]

You might also look into the Hosmer Lemeshow gof test, etc:

http://www.stata-journal.com/sjpdf.h...iclenum=st0099

http://www.statalist.org/forums/foru...on-survey-data

[Steve Samuels]

estat gof after svy: logistic is the Hosmer-Lemeshow test of fit.

[David Radwin]

Percent correctly predicted is not necessarily a particularly good indication of goodness-of-fit, particularly for samples where successes or failures are rare. For example, if only 0.01% of applicants are accepted to a school, the model "nobody is accepted" (a constant with no parameters) would yield 99.99% correct predictions.

One alternative goodness-of-fit measure:
Herron, M. C. (1999). Postestimation uncertainty in limited dependent variable models. Political Analysis, 8(1), 83-98. http://www.polmeth.wustl.edu/analysis/vol/8/herron.pdf