Test for paired multiple proportions

[Jake Magel]

Hello,


I have pre-post test data on the proportions of participants who chose 1 category among several response options (below). I would like to test whether the proportions of responses are different between pre and post-test. I am unsure which test to use when testing multiple proportions as outlined below. I would appreciate advice on which test to use and the code to complete the test. Or advice on where I can find the information to figure it out as I have not been able to find it using Stata help. Thanks!

Never
Rarely - less than 10% of the time
Occasionally - about 30% of the time
Sometimes - about 50% of the time
Frequently - about 70 % of the time
Usually - about 90% of the time
Everytime

* Example generated by -dataex-. For more info, type help dataex
clear
input byte record_id float time_rc byte(freq1_ask_fu1 freq2_screen_fu1 freq3_assess_fu1 freq4_advise_fu1 freq5_refer_fu1)
 1 0 1 0 0 0 0
 1 1 3 1 2 2 0
 1 2 4 3 4 4 2
 2 0 0 0 0 0 0
 2 1 1 1 1 0 0
 2 2 1 0 0 1 0
 3 0 0 0 0 0 0
 3 1 1 0 3 2 1
 3 2 1 0 2 1 1
 4 0 0 0 1 1 0
 4 1 2 0 2 1 0
 4 2 2 1 3 2 0
 5 0 1 0 0 1 0
 5 1 3 0 2 1 0
 5 2 3 1 1 1 0
 6 0 1 0 0 1 1
 6 1 3 3 3 3 1
 6 2 5 5 5 6 3
 7 0 1 0 1 1 1
 7 1 2 1 2 2 2
 7 2 2 1 1 1 2
 8 0 1 0 1 3 0
 8 1 2 1 3 4 1
 8 2 2 1 2 4 1
 9 0 1 0 0 1 0
 9 1 2 0 2 1 0
 9 2 3 0 1 2 0
10 0 0 0 1 0 0
10 1 1 0 1 0 0
10 2 1 0 1 0 .
11 0 1 0 0 0 0
11 1 4 4 4 4 1
11 2 3 3 3 2 0
12 0 1 0 1 1 0
12 1 4 3 3 2 1
12 2 5 1 1 1 3
13 0 2 0 4 3 1
13 1 3 1 5 4 2
13 2 3 0 5 5 1
end
label values freq1_ask_fu1 freq1_ask_fu1_
label def freq1_ask_fu1_ 0 "Never", modify
label def freq1_ask_fu1_ 1 "Rarely - less than 10% of the time", modify
label def freq1_ask_fu1_ 2 "Occasionally -  about 30% of the time", modify
label def freq1_ask_fu1_ 3 "Sometimes - about 50% of the time", modify
label def freq1_ask_fu1_ 4 "Frequently - about 70% of the time", modify
label def freq1_ask_fu1_ 5 "Usually - about 90% of the time", modify
label values freq2_screen_fu1 freq2_screen_fu1_
label def freq2_screen_fu1_ 0 "Never", modify
label def freq2_screen_fu1_ 1 "Rarely - less than 10% of the time", modify
label def freq2_screen_fu1_ 3 "Sometimes - about 50% of the time", modify
label def freq2_screen_fu1_ 4 "Frequently - about 70% of the time", modify
label def freq2_screen_fu1_ 5 "Usually - about 90% of the time", modify
label values freq3_assess_fu1 freq3_assess_fu1_
label def freq3_assess_fu1_ 0 "Never", modify
label def freq3_assess_fu1_ 1 "Rarely - less than 10% of the time", modify
label def freq3_assess_fu1_ 2 "Occasionally -  about 30% of the time", modify
label def freq3_assess_fu1_ 3 "Sometimes - about 50% of the time", modify
label def freq3_assess_fu1_ 4 "Frequently - about 70% of the time", modify
label def freq3_assess_fu1_ 5 "Usually - about 90% of the time", modify
label values freq4_advise_fu1 freq4_advise_fu1_
label def freq4_advise_fu1_ 0 "Never", modify
label def freq4_advise_fu1_ 1 "Rarely - less than 10% of the time", modify
label def freq4_advise_fu1_ 2 "Occasionally -  about 30% of the time", modify
label def freq4_advise_fu1_ 3 "Sometimes - about 50% of the time", modify
label def freq4_advise_fu1_ 4 "Frequently - about 70% of the time", modify
label def freq4_advise_fu1_ 5 "Usually - about 90% of the time", modify
label def freq4_advise_fu1_ 6 "Every Time", modify
label values freq5_refer_fu1 freq5_refer_fu1_
label def freq5_refer_fu1_ 0 "Never", modify
label def freq5_refer_fu1_ 1 "Rarely - less than 10% of the time", modify
label def freq5_refer_fu1_ 2 "Occasionally -  about 30% of the time", modify
label def freq5_refer_fu1_ 3 "Sometimes - about 50% of the time", modify

[Joseph Coveney]

I would like to test whether the proportions of responses are different between pre and post-test.

You have three time points. So the latter two are what you mean by posttest?

In your representation of the choices, the underlying difference between categories is roughly the same, and if that sticks with your participants when they fill out the questionnaire, then how about a conventional linear model? Something like the following.

rename (record_id time_rc) (rid tim)
rename freq* sco#, renumber
quietly reshape long sco, i(rid tim) j(itm)

mixed sco i.itm##i.tim || rid: , reml dfmethod(kroger) nolrtest nolog
testparm i.itm#i.tim, df(`e(ddf_m)')
testparm i.tim, df(`e(ddf_m)')
margins tim

predict double res, residuals
set more on
// Using this approach hinges on whether you're good with this:
qnorm res, scheme(s2color) mcolor(black) msize(vsmall) ///
    ylabel( , angle(horizontal) nogrid)
more
set more off

/* If not, then -help permute-

   . . . or fit an analogous large-sample ordered-categorical regression model,
   which in your case yields the same conclusion: */
meologit sco i.itm##i.tim || rid: , nolog
testparm i.itm#i.tim
testparm i.tim

(I've shortened the variable names for convenience and for legibility in the regression table.)

[Jake Magel]

HI Joseph, thanks so much. Very helpful!