​Effect size for ordered logit models (ologit)

[Sebastian Sattler]

Dear Statalist members,
I would like to calculate effect sizes (such as Cohen´s d) after running several ordered logit models with different dependent variables (each with 11 dichotomous and 4 “metric/quasi-metric” variables). Side note: The models are calculated with multiple imputed (mi) data.
Any suggestion would be highly appreciated.
Thank you very much!

[Clyde Schechter]

Can you expand on what you want to do here?

Cohen's d is generally used as a measure of effect size when you have a dichotomous predictor and a continuous outcome--it is based on standard deviations of the outcome. With order logit, you have an ordinal outcome variable, so standard deviation isn't even defined for it. Probably the most commonly used measure of effect size for dichotomous predictors and ordinal outcome when the proportional odds assumption holds (which is presumed with -ologit-), is the odds ratio. You can calculate the odds ratio by exponentiating the regression coefficient. For continuous predictors, the most common I have seen is the odds ratio associated with a 1 unit increase in the predictor.

[Sebastian Sattler]

Thank you very much for your fast response. I am familiar with odds ratios.
I was hoping that there might be a way to get a more standardized output than ORs (or percent change in the odds).

[Laura Myles]

Hi Clyde,

Any suggestion on how to obtain 1 effect size for a an ordinal logistic regression when I have 2 predictors plus their interaction. One predictor is binary but the other has 6 levels. Should I just report all ORs or is there a way to obtain an overall effect size for the predictor with 6 levels as well as the interaction.

The testparm command only shows the omnibus significance level.

Thanks.

[Mike Lacy]

One thought would be to report an increment to the R² when this variable is added last to the model. See -ssc describe r2o-, which a command I wrote to compute an ordinal R² I developed (explained dispersion w/o supra-ordinal assumptions.) This measure is independent of sample size. The article cited in the help for that command documents the measure and reports a Monte Carlo study of several ordinal R² measures, which are compared with respect to their ability to reproduce a conventional R² for an underlying continuous variable. You could also try some of the other pseudo-R² available in the -fitstat- command of the -spost13- package.

[Weiwen Ng]

Chen and colleagues proposed here that in binary regressions, an OR around 1.6 may be a small effect size (~ 0.2 Cohen's d), 3.5 may be medium, and 6.7 may be large. In ordered logit, you're estimating the log odds for responding in a higher category. I suspect you can apply the same logic that Chen et al.

Mike's thought about incremental pseudo-R² is conceptually akin to Cohen's eta-squared effect size measure. In other words, eta-squared measures the incremental R² from one additional predictor. I'm not sure how well it translates to the pseudo-R²s in logistic regression, though.