I'm studying whether an institution changes respondents' answer to a dichotomous question, and hence using a chi2 (or fisher's) test to see if I can reject the null that those distributions are independent. 191 people answered to the survey, half of them saw the questions without the institution first, and half saw the questions with the institution first. All 191 answered to all questions.
My questions is: in question 1 (without the institution), 58% answered 1, and in question 2 (with the institution) 18% answered 1, and the p-value of this very large difference is 0.282.
But in question 3 (without the institution), 28% answered 1, and in question 4 (with the institution) 13% answered 1, and the p-value of this much smaller difference is 0.019.
Can this happen or can I be sure I'm doing something wrong? Is it because of who changed his or her own answer, and hence because of the changes in the contingency table?
tab AnswerQ1 AnswerQ2 if usa==1 & Treat==1, chi2
| Answer.Q2
Answer.Q1 | 0 1 | Total
-----------+----------------------+----------
0 | 69 12 | 81
1 | 87 23 | 110
-----------+----------------------+----------
Total | 156 35 | 191
Pearson chi2(1) = 1.1576 Pr = 0.282
tab AnswerQ3 AnswerQ4 if usa==1 & Treat==1, chi2
| Answer.Q4
Answer.Q3 | 0 1 | Total
-----------+----------------------+----------
0 | 124 13 | 137
1 | 42 12 | 54
-----------+----------------------+----------
Total | 166 25 | 191
Pearson chi2(1) = 5.5204 Pr = 0.019
thanks a lot,
My questions is: in question 1 (without the institution), 58% answered 1, and in question 2 (with the institution) 18% answered 1, and the p-value of this very large difference is 0.282.
But in question 3 (without the institution), 28% answered 1, and in question 4 (with the institution) 13% answered 1, and the p-value of this much smaller difference is 0.019.
Can this happen or can I be sure I'm doing something wrong? Is it because of who changed his or her own answer, and hence because of the changes in the contingency table?
tab AnswerQ1 AnswerQ2 if usa==1 & Treat==1, chi2
| Answer.Q2
Answer.Q1 | 0 1 | Total
-----------+----------------------+----------
0 | 69 12 | 81
1 | 87 23 | 110
-----------+----------------------+----------
Total | 156 35 | 191
Pearson chi2(1) = 1.1576 Pr = 0.282
tab AnswerQ3 AnswerQ4 if usa==1 & Treat==1, chi2
| Answer.Q4
Answer.Q3 | 0 1 | Total
-----------+----------------------+----------
0 | 124 13 | 137
1 | 42 12 | 54
-----------+----------------------+----------
Total | 166 25 | 191
Pearson chi2(1) = 5.5204 Pr = 0.019
thanks a lot,
