Hosmer-Lemeshow test

[Janio Mossinato]

Hi I'm trying to use the command sequence -estat gof- to do the Hosmer-Lemesshow test after a xtlogit, however, the result is: . estat gof invalid subcommand gof r(321); Could you help me to clarify the type of error?

[Clyde Schechter]

-estat gof- only runs after -logit- or -probit-, not their -xt- versions.

You can emulate the Hosmer-Lemeshow procedure after -xtlogit, re- "by hand" in the following way:

predict predicted, pr
xtile decile = phat, nq(10) // OR DO AN APPROPRIATE NUMBER OF GROUPS
collapse (sum) observed = outcome phat, by(decile)

You now have a data set with one observation per decile of predictive risk, with the number of observed and predicted outcomes. You can graph these if you like or list them to see a table. Inspecting this will give you an assessment of your model's calibration. If your data set is large, it probably makes sense to use more than just 10 groups of predicted risk. If your sample is very small, then use fewer groups. If all of the model's predictive variables are categorical, then you should use the Pearson approach instead (combinations of predictors, rather than quantiles of predicted risk).

You could also calculate a "chi square statistic" as the sum of (observed - expected)^2/expected. But that statistic would not have a chi square distribution because the observations that went into calculating observed and expected are not independent. I personally don't think that summary chi square statistic is useful all that often anyway.

If you have run -xtlogit, fe-, then there is nothing analogous to the Hosmer-Lemeshow analysis because -xtlogit, fe- does not provide predicted probabilities.

[Janio Mossinato]

Dear Clyde
Tank you for your assistance.
Jânio

[Susan Wu]

-estat gof- only runs after -logit- or -probit-, not their -xt- versions.

You can emulate the Hosmer-Lemeshow procedure after -xtlogit, re- "by hand" in the following way:

predict predicted, pr
xtile decile = phat, nq(10) // OR DO AN APPROPRIATE NUMBER OF GROUPS
collapse (sum) observed = outcome phat, by(decile)

You now have a data set with one observation per decile of predictive risk, with the number of observed and predicted outcomes. You can graph these if you like or list them to see a table. Inspecting this will give you an assessment of your model's calibration. If your data set is large, it probably makes sense to use more than just 10 groups of predicted risk. If your sample is very small, then use fewer groups. If all of the model's predictive variables are categorical, then you should use the Pearson approach instead (combinations of predictors, rather than quantiles of predicted risk).

You could also calculate a "chi square statistic" as the sum of (observed - expected)^2/expected. But that statistic would not have a chi square distribution because the observations that went into calculating observed and expected are not independent. I personally don't think that summary chi square statistic is useful all that often anyway.

If you have run -xtlogit, fe-, then there is nothing analogous to the Hosmer-Lemeshow analysis because -xtlogit, fe- does not provide predicted probabilities.

Then, is there any way we can test the fitness of the model after using -xtlogit,fe- ??

[Clyde Schechter]

I don't know of any way to do this. Perhaps somebody else does and will chime in.

[Kazim Hashim]

Hello,

I am currently running a probit regression to investigate the likelihood of entering higher education based on a few predictors. Can I run a Hosmer-Lemeshow test in stata as a goodness of fit test? if not, what tests are suitable.

[Clyde Schechter]

Yes. The command is -estat gof- after the probit. Run -help probit postestimation- and then click on the estat gof link for details of the options you can specify..

[Kazim Hashim]

Thank you Clyde!

[Shilpa Shetty]

Hello,

I am running logistic regression for panel data (xtlogit, re). Hosmer Lemeshow test result is not generated by Stata. How can I test the goodness of fit of the developed model? Is there any other alternative to Hosmer Lemeshow test? Kindly guide.