Using ORs incl. 95%CIs to find pairwise differences

[Sara Hansen]

I am investigating how students score on a test, based on their experience on the subject. My outcome, test_answer, is binary (0 for a wrong answer, 1 for a correct answer). Experience is a categorical variabel: group 0-3 (0 is no experience, and 3 is the group with the most experience). I have investigated this using a logistic regression model:

logit test_answer i.experience, or

Thus, I’ve got ORs incl. 95% CIs and p-values for all experience-groups compared to no experience:

• Experience-group 0: ref.
• Experience-group 1: 4.5 (4.1-4.9)
• Experience-group 2: 9.1 (7.4-11.0)
• Experience-group 3: 8.0 (7.1-8.9)

(All p<0.001)

I would like to also compare the groups with each other, and not just with the reference group. I could use chi^2 Test of Independence to determine whether there is a significant association between reader experience and test-score, and this way see if there is pairwise differences between them.

However, I was wondering if there is a more elegant way to do it. I already have ORs incl 95% CIs for the groups, so I think, that I somehow could make use of these to get 95%CIs for the pairwise differences, and from these calculate p-values. But I am not sure how I would do this. Can any of you guys help me? Or maybe you have an even better/more elegant way?

Additional info: My data format is long, i.e. like this:

student_id     experience     test_answer   

1              0              1

1              0              0

1              0              1

...            ...            ...

2              2              0

2              2              1

...            ...            ...

[Rich Goldstein]

what you want is not completely clear to me but if you look at

help logit postestimation

and click on either or both of "contrast" and "pwcompare", I think you find what you need

[Sara Hansen]

Thank you, Rich Goldstein.
However, I am not sure if this is appropriate for my data, since my outcome is binary, and this produces pairwise comparisons of marginal linear prediction?

[Joseph Coveney]

I am not sure if this is appropriate for my data, since my outcome is binary, and this produces pairwise comparisons of marginal linear prediction?

A logistic regression model is one kind of generalized linear model, and the comparisons that contrast and pwcompare perform after fitting your logistic regression model are appropriate.

There is one feature of your dataset, though, that calls into question whether logit is appropriate for it: you seem to have multiple observations per student. Wouldn't you want to fit some kind of multilevel or hierarchical logistic regression model in order to accommodate that? Maybe something like

melogit test_answer i.experience i.test_question_id || student_id:
pwcompare experience, mcompare(bonferroni)