Does the significance of cutpoints (thresholds) matter in ordered logit or ordered probit models?

[Al Etzdy]

I have used ordered logit in Stata to model an ordered DV with 5 categories. Out of the four cutpoints, one is insignificant (coefficient value <= 2*S.E). i wonder if I need to collapse some categories, and if not, whether my model estimates are biased.

[Richard Williams]

First off, there is no reason a cut point couldn't be zero.

I think it is interesting that ologit doesn't even report the Z values for cut points. Further, I've never seen anybody try to do much with the cutpoints.

If I had really small Ns for some categories, I might consider combining them. But just based on what you say, I don't see any need to combine.

[Al Etzdy]

Thanks Richard. I agree that there is no problem in having a "0" cutpoint. However, Couldn't one say that the insignificant cutpoint that is technically statistically zero, but according to the results shows to be, for instance, 0.4 would cause error in estimation? in other words, cutpoint "0" should have been used by Stata to differentiate the categories, but now "0.4" is used. What do you think?

[Maarten Buis]

The estimate of the cut point, your 0.4, is the best estimate given your data and model. Setting your estimate to 0, would make the model fit worse. Not by a lot, and we have difficulties statistically differentiating between the models, but still worse. Now it is a tradeoff, which will involve considerations like how simple do you want your model to be, how much information is present in the data, etc. etc.

[John Mullahy]

Richard's point in #2 about the small-N-cell case seems important.

Such an instance should translate into adjacent cutpoints being "close" in some sense since they are defining a low-probability cell. (Picture the integral between two nearby points under the standard normal density.)

If you were going to do hypothesis testing with a goal of possibly collapsing cells this may be worth considering. That is, differences between adjacent cutpoints are informative about the (conditional) probability structure of your ordered outcome.

That said, such a test may be a bit tricky to conduct since the cutpoints necessarily obey an ordering relationship and aren't fully free parameters; as such the relevant alternative hypothesis might be a "less than" rather than an "unequal to" relationship.

[Richard Williams]

in other words, cutpoint "0" should have been used by Stata to differentiate the categories, but now "0.4" is used. What do you think?

You because an estimate doesn't significantly differ from 0 doesn't mean it is zero. Indeed, it could go the other way. It doesn't significantly differ from zero, but it also doesn't significantly differ from some larger value, e.g. in your case the .4 estimate may not differ significantly from 0 but it also may not significantly differ from 1. As Maarten say, .4 is the best guess as to what the value really is.

[Al Etzdy]

Thanks all for their help.
I agree with all the points mentioned here, but what partly confused me was when I came across this statement in (Garson, G. David. "Ordinal Regression (Statistical Associates "Blue Book" Series Book 9)", Statistical Associates Publishers. Asheboro, NC.):
“Unless nonsignificant, threshold values generally are not important to interpretation of study results, but instead represent simple cutoff points. Generally, threshold values are significant and different, as they are in this example. If a threshold is non-significant, then it cannot be concluded to differ from zero, meaning that effectively that that level and the one above have the same equations (since by the parallelism assumption, location slopes are the same and only thresholds differ). Put another way, non-significance of a threshold suggests that the cutting point is not truly different and therefore some levels of the dependent variable need to be combined."
Not sure if I agree with what is stated in this book, but would like to have you guys' opinion on it as well to be sure.

[Richard Williams]

If a threshold is non-significant, then it cannot be concluded to differ from zero, meaning that effectively that that level and the one above have the same equations (since by the parallelism assumption, location slopes are the same and only thresholds differ). Put another way, non-significance of a threshold suggests that the cutting point is not truly different and therefore some levels of the dependent variable need to be combined."

As worded, I do not agree with that. Just because a threshold does not significantly differ from 0 does not mean that it doesn't significantly differ from the threshold above (or below) it. You need to use the test command or something like that to test if two thresholds differ. And this could be true of any adjoining thresholds, e.g. thresholds of 3 and 3.1 might not significantly differ. I see no reason to focus on thresholds where one of the thresholds does not significantly differ from 0.

Even if two thresholds do not significantly differ, I don't see why you have to combine thresholds. It might make the model more parsimonious if you do, but parsimony is generally not critical.

Again I might be tempted to combine if categories have thin Ns. But nothing in what you have described so far makes me feel an urge to combine categories in this case.