McNemar or paired testing help

[Alex Joseph]

Hi all,

Looking for big picture guidance.

I am using a dataset that looks at outcomes of an eye procedure done to both eyes in one patient. In each patient, one eye has had a prior surgery and is may be at a higher risk in the second surgery. I wanted to compare the eye with prior operation vs eye with no prior operation within each patient. The small n is just 9 patients, so 18 eyes.

Outcomes are binary yes/no.

Options would be just a fishers exact. Could I apply McNemars here?

Mixed effects logistic regression could also be an option, controlling for individual patient, but #s are very small again.

Thank you for suggestions.

[Mike Lacy]

Your N is way too small to trust any logit results, let alone mixed logit. The McNemar test would be quite reasonable, and it can be used (and commonly is) with an exact binomial test, which would be nice for the sample size. (In fact, if I recall correctly, one of the classic data sets for illustrating the McNemar test involves a sample with two eyes per subject.)
However, while the McNemar can be quite valid here, you won't have much precision with N = 9 pairs.

One note: There's a strong preference for real names here on StataList. See https://www.statalist.org/forums/help#realnames.

[Alex Joseph]

Thanks, Mike. I am new and will update name! I have one follow-up. I see the option for an exact McNemar. Since my N is so low and counts obviously less than 5, should I plan to use this, as with Fisher's for a chi square?

[Mike Lacy]

Yes. Exact tests do what asymptotic test try to approximate. There are some technical issues and a substantial literature on various variations on the best small-sample tests, and some argue for some adjusted versions of asymptotic tests being better, but at least we can say here that an exact binomial test with N = 9 will be better than the plain-vanilla asymptotic test you'd otherwise get from any of the various default ways to get a McNemar test.

[Alex Joseph]

Thank you for quick response! Make that two questions for now (I won't box myself in)..

Something must be wrong with the way I structured the data?

I simply have one column to indicate yes/no on prior surgery and another column with whether a complication occurred.

In this example, I had two patients with prior surgery have a complication while there were no complications in group without prior surgery.

----

mcc priorsurgery complication


| Controls |
Cases | Exposed Unexposed | Total
-----------------+------------------------+------------
Exposed | 2 7 | 9
Unexposed | 0 9 | 9
-----------------+------------------------+------------
Total | 2 16 | 18

McNemar's chi2(1) = 7.00 Prob > chi2 = 0.0082
Exact McNemar significance probability = 0.0156

Proportion with factor
Cases .5
Controls .1111111 [95% Conf. Interval]
--------- --------------------
difference .3888889 .1081249 .6696529
ratio 4.5 1.325547 15.27671
rel. diff. .4375 .1944261 .6805739

odds ratio . 1.441309 . (exact)


Additionally, I run the same test on complication2, and get a significant p even though I had one complication in both cohorts.

mcc priorsurgery complication2


---

| Controls |
Cases | Exposed Unexposed | Total
-----------------+------------------------+------------
Exposed | 1 8 | 9
Unexposed | 1 8 | 9
-----------------+------------------------+------------
Total | 2 16 | 18

McNemar's chi2(1) = 5.44 Prob > chi2 = 0.0196
Exact McNemar significance probability = 0.0391

Proportion with factor
Cases .5
Controls .1111111 [95% Conf. Interval]
--------- --------------------
difference .3888889 .0605119 .7172659
ratio 4.5 1.125439 17.99297
rel. diff. .4375 .1618801 .7131199

odds ratio 8 1.072554 354.9812 (exact)

.
Sorry, there must be a better way to import tabular data like that and keep the structure, I am new. I am obviously a novice here and appreciate any help or insight!

Thanks.

[Mike Lacy]

I can't say anything about how you structured the data because you have not provided an example of the data so we can know that (Search the FAQ for -dataex- and you can learn how to display data nicely and easily on StataList. You have such a small data set that you can display the whole thing.). Also, I now am not sure I have understood your data situation and your question. Is the situation this: Each patient has had two surgeries. On each surgery, you observe whether s/he had a complication, yes/no.
And, is the question: "Are patients who had a complication on the first surgery on Eye1 more likely to have a complication on the second surgery on Eye2 than patients who did not have a complication on the first surgery?" If that's true, you don't want the McNemar test and I have misunderstood you. I would need to know more clearly what each variable means and what the study design was to give you a better answer.

And, lastly: As you can see, your output on the screen is not readable and can't easily be fixed to make it readable. That problem can be solved in the future by pasting output text into StataList as text, highlighting it, and then clicking on the "#" icon, which will make it display in an appropriate way in your StataList posting.

[Alex Joseph]

Thanks, Mike. I appreciate your patience and willingness to teach. I will look into how to provide the dataset and report back.

This is a retrospective cohort, with patients who had surgery on one eye, but not the other initially. Then, they ended up having a second surgery on both eyes during the second operation. My question is, is the eye that was previously operated on at a higher risk for complications compared to the other eye, which has not been operated on previously. I thought it would be the best internal control to compare eyes within the same individual where the only difference is a prior intervention (what I am after).

I will be sure to format the output appropriately next time. Thanks for teaching.

[Alex Joseph]

* Example generated by -dataex-. To install: ssc install dataex
clear
input byte(patient priorsurgery complication)
1 1 1
2 1 0
3 1 1
4 1 0
5 1 0
6 1 0
7 1 1
8 1 0
9 1 0
1 0 0
2 0 0
3 0 0
4 0 0
5 0 0
6 0 0
7 0 0
8 0 0
9 0 0
end

[Alex Joseph]

Id imagine a potentially more appropriate structure for this would be:

* Example generated by -dataex-. To install: ssc install dataex
clear
input byte(patientID controleye_complication caseye_complication)
1 0 0
2 0 1
3 0 1
4 0 0
5 0 0
6 0 0
7 0 0
8 1 1
9 0 0
end

[Mike Lacy]

Your last presentation makes the situation clear, and indeed, the McNemar test is appropriate: You desire a test to compare proportions on the same variable "Was there a complication?," where those proportions are obtained from the same individuals but in different contexts/times/conditions.

* Example generated by -dataex-. To install: ssc install dataex
clear
input byte(patientID controleye_complication caseye_complication)
1 0 0
2 0 1
3 0 1
4 0 0
5 0 0
6 0 0
7 0 0
8 1 1
9 0 0
end
. tab controleye_complication caseeye_complication

controleye |
_complicat |  caseye_complication
       ion |         0          1 |     Total
-----------+----------------------+----------
         0 |         6          2 |         8
         1 |         0          1 |         1
-----------+----------------------+----------
     Total |         6          3 |         9

The preceding reveals 2 individuals with discordant responses, and the insight behind the McNemar test is that the relevant difference of proportions rests only on the distribution of discordant responses, with the concordant ones being irrelevant. Of the N = 2 discordant responses, 2 are 0/1 and 0 are 1/0. The McNemar test can then be obtained as

bitesti 2 2 0.5

        N   Observed k   Expected k   Assumed p   Observed p
------------------------------------------------------------
        2          2            1       0.50000      1.00000

  Pr(k >= 2)           = 0.250000  (one-sided test)
  Pr(k <= 2)           = 1.000000  (one-sided test)
  Pr(k <= 0 or k >= 2) = 0.500000  (two-sided test)

The three Pr listings give you the p-value under each possible alternative hypothesis.