Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Fisher Exact Test: p-value discrepency stata vs formula

    Hey everyone,
    this forum has been a big help with prior stata problems I had. This is the first time I could not find an answer to my question on a different post. I have two datasets and conducted a Fisher Exact Test using stata 18 with the following code:



    Code:
    cci 59 55 52 62, exact
    Code:
    cci 13 8 13 17, exact
    This results in the following output tables:
    Code:
                                                             Proportion
                     |   Exposed   Unexposed  |      Total      exposed
    -----------------+------------------------+------------------------
               Cases |        59          55  |        114       0.5175
            Controls |        52          62  |        114       0.4561
    -----------------+------------------------+------------------------
               Total |       111         117  |        228       0.4868
                     |                        |
                     |      Point estimate    |    [95% conf. interval]
                     |------------------------+------------------------
          Odds ratio |         1.279021       |    .7354733    2.225323 (exact)
     Attr. frac. ex. |          .218152       |   -.3596687    .5506271 (exact)
     Attr. frac. pop |         .1129032       |
                     +-------------------------------------------------
                                      1-sided Fisher's exact P = 0.2133
                                      2-sided Fisher's exact P = 0.4267
    Code:
                                                             Proportion
                     |   Exposed   Unexposed  |      Total      exposed
    -----------------+------------------------+------------------------
               Cases |        13           8  |         21       0.6190
            Controls |        13          17  |         30       0.4333
    -----------------+------------------------+------------------------
               Total |        26          25  |         51       0.5098
                     |                        |
                     |      Point estimate    |    [95% conf. interval]
                     |------------------------+------------------------
          Odds ratio |            2.125       |    .5926572    7.795925 (exact)
     Attr. frac. ex. |         .5294118       |    -.687316    .8717279 (exact)
     Attr. frac. pop |         .3277311       |
                     +-------------------------------------------------
                                      1-sided Fisher's exact P = 0.1536
                                      2-sided Fisher's exact P = 0.2581
    Afterwards I used the data to calculate the p-values by hand using the formula from https://en.wikipedia.org/wiki/Fisher%27s_exact_test. The resulting p-value for the first dataset was p=0.0688 and for the second example I calculated p=0.983. Overall I cannot report significant results at the 5% level either way but I would like to understand where the discrepency comes from.

    I have already looked at https://www.stata.com/support/faqs/s...rs-exact-test/ but think it does not explain my problem but it could be that I misunderstood the post.

    Thank you for taking the time to read my post (:
    Why does Fisher's exact test disagree with the confidence interval for the odds ratio?

  • #2
    Have you checked the Methods and Formulas section of the command's PDF manual entry for details on Stata's calculation? The Stata FAQ that you link admits to different ways to calculate the p-value, so there is no global "correct" way of doing this.

    Comment


    • #3
      Thank you for the reply. I checked the section and it is really short but I compared it a lecture slides on the Fisher Exact Test from the university of chicago and am pretty sure that the concept is the same. Following the Stata FAQ I assumed that even if there are multiple ways to calculate the p-value for the two sided test they should be resulting in similar values. In this example the difference is rather extreme resulting in a difference large enough for one method resulting in a p-value that is significant at the 10% level and one that is not significant at any level. Further I tried using the formula to calculate the p-values of some other stata example i found in the forum and got very similar or even the exact same p-values with stata. Which still leaves the question of why my two datasets result in such large differences

      Comment


      • #4
        Originally posted by Lennart Rehm View Post
        Following the Stata FAQ I assumed that even if there are multiple ways to calculate the p-value for the two sided test they should be resulting in similar values.
        No, that's not correct. Consider the following statements:

        The test naturally gives a one-sided p-value, and there are at least four different ways to convert it to a two-sided p-value (Agresti 2002, 93). One way, not implemented in Stata, is to double the one-sided p-value; doubling is simple but can result in p-values larger than one.

        Stata instead adds the probabilities of all the tables at least as unlikely as the observed table. (For a rigorous statement, see Methods and Formulas of [R] tabulate twoway). In our example, all the “unlikelier” tables are in the same tail as the observed table. The other tail does not contribute to the p-value, so the one-sided and two-sided p-values are equal.
        So Stata in this instance reports the same one-sided and two-sided p-value (see highlighted), whereas an alternative calculation reports the doubled one-sided p-value.

        Comment


        • #5
          Thanks again. I think i found the mistake in my manual approach. I will check it and might ask more questions if it still doesn't work

          Comment


          • #6
            Check this paper:

            https://academic.oup.com/ije/article/37/6/1430/731900

            Comment

            Working...
            X