Pearson Goodness of Fit with xtprobit

[Euan Ritchie]

Dear Statalist,

I am running a dynamic random effects probit model using xtprobit, but I need to find the Pearson goodness of fit. Whilst this is easy for probit ("estat gof") this command is not allowed for xtprobit. In fact the postestimation options for xtprobit seem rather limited. However I have seen the Pearson goodness of fit measure quoted for similar models. I have of course consulted the manuals but I could not find any information on how to obtain it. In addition, this question has been asked before on Statalist but were unanswered.

The model is "xtprobit belnmw lbel bel0 edu x1 x2 health mastat sex lonse scot wales ireland $averages"

Would be grateful for help,

[Nick Cox]

Usually when such measures are not provided the reasons are either or both (1) the measure is not defined in the literature (2) it is hard to think of it as being unproblematic.

In a panel context, the difficulty is often accounting for the dependence structure of a panel model. Most people asking for this kind of extension don't really want a measure; they want a P-value. (Naturally, your stance may be different.)

If you can cite literature discussions or provide independent motivation for such a measure that might boost this on the StataCorp to-do list.

Alternatively, telling us exactly how (you think) the measure would be defined would make it easier for experts in the field (not me) to comment and/or provide advice on how to calculate it.

[Euan Ritchie]

Nick Cox - Thanks for your reply. The main paper that quotes such a measure is one of the seminal papers in this literature:

"Stewart, M.B., 2007. The interrelated dynamics of unemployment and low-wage employment.
Journal of Applied Econometrics 22, 511–531"

In the paper, the dependent variable is a binary and there are six waves of the panel data. Therefore there are 2^6 64 different possible sequences. In the paper, the Pearson GoF statistic is calculated by predicting the amount of observations in each of the 6 "bins" and comparing that to the actual amount. This is compared to a chi^2 distribution of 63 degrees of freedom.

I understood this to be the standard way the statistic was used but I'm not entirely familiar with it so apologies if not. Certainly it seems an intuitive measure of how well the model fits.

[Nick Cox]

Thanks for the detail. If it's a standard chi-square test you want, then that's easy enough through commands such as tabulate or tabi. I can't comment expertly on how that relates to the extra structure of an xtprobit model. But in essence the chi-square calculation can't account for structure it's not told about, so while you can calculate chi-square what the appropriate degrees of freedom are and whether standard chi-square distributions apply is a different ball game.

I really would stick with the diagnostics xtprobit provides rather than adding home-grown measures.

[Stephen Jenkins]

My understanding the chi-squared test that Mark Stewart implements is the 'standard' one, as Nick intimates. To me, the much harder problem is to generate the predicted sequences to compare with the observed ones. How did you do that?

[Euan Ritchie]

Stephen - I'm afraid I haven't, I suppose I hoped that's what there might have been a command for. Although if anyone could provide help on how to do so it would be greatly appreciated. Otherwise, thanks again for your responses and I will make do with the inbuilt commands as Nick suggests, and attempt to develop my stata ability in the long run.