Compare goodness of fit between an ordered probit and OLS

[Martin Orr]

What goodness of fit statistics would be valid to compare a model running my data with OLS and a model using an ordered Probit. I was going to simply use log-likelihood and also an adjusted R squared provided by the fitstat command, I was wondering if it was valid to compare OLS and an ordered probit using these values. The log likelihood is smaller with the ordered probit but the adjusted r squared provide conflicting results with the Mcfadden being higher for the ordered Probit and then McFadden adjusted, Cox-Snell, and Cragg-Uhler being lower for OLS. Any help would be greatly appreciated.

[Phil Bromiley]

You didn't get a quick answer. You'll increase your chances of a useful answer by following the FAQ on asking questions.

I'm not sure this is a meaningful comparison. As I understand it,the ordered probit gives an equation predicting y with cutpoints. But the ols doesn't give cutpoints. I don't think you can just use fitstat or log likelihood. So, you'll be forced to impose some rule to generate predictions. If you generate the cutoffs for ols, then you could calculate any of a variety of fit statistics.

[Nick Cox]

Also posted on https://stats.stackexchange.com/ques...ordered-probit

My comments there I think match, or are consistent with, Phil's above.

I'd add a caution that not all beasts called R-square are generally comparable, particularly across different models. Weasel words like adjusted or even pseudo- don't prepare the unwary for objects defined in quite different ways. At most what they have in common is that values don't fall outside [0, 1]. It's not even guaranteed that the bounds of 0 and 1 are attainable.

[Weiwen Ng]

I'll just add that because the outcome is ordered categorical, most reviewers in most disciplines would probably not question why you didn't use OLS over an ordered logit/probit model. OLS is most properly used on a continuous outcome. Your outcome is not continuous. I don't think an econ or statistical journal would accept the article at all.

Log likelihood is valid for comparing nested models. I can't quite see the ordered logit/probit being nested in an OLS model. Different link function, different distributional family. Hence, I don't think the log likelihood can be compared across the two models.