Bonferroni Correction with Dunn Test

[Lucy Kay]

What exactly does the Bonferroni correction in the Dunn Test code do:

 dunntest rating, by(outcome) ma(bonferroni)

With the regular two-sample Wilcoxon rank-sum (Mann-Whitney) test, you need to multiple the p-value by the number of groups to get the Bonferroni corrected p-value. Does the Dunn test already do this for us?

[Nick Cox]

Cross-posted at https://www.reddit.com/r/stata/comme...rrection_code/

I don't understand the premise. Are you comparing repeating the Mann-Whitney test again and again for all possible pairs?

More crucially, dunntest here refers to a command from the Stata Journal -- as you are asked to explain -- which is all about doing multiple comparisons in a statistically responsible way Make sure you use the updated version.

SJ-15-2 st0381_1 . . . . . . . . . . . . . . . . Software update for dunntest
(help dunntest if installed) . . . . . . . . . . . . . . . . A. Dinno
Q2/15 SJ 15(2):605--606
adds new features: Hochberg's adjustment for multiple
comparisons, Benjamini-Yekutieli false-discovery-rate
adjustment for multiple comparisons, and option to report
row-column or column-row pairwise differences; corrects
several bugs

SJ-15-1 st0381 Nonparametric pairwise multiple comparisons using Dunn's test
(help dunntest if installed) . . . . . . . . . . . . . . . . A. Dinno
Q1/15 SJ 15(1):292--300
produces nonparametric pairwise multiple comparisons following
a Kruskal-Wallis k-way test (kwallis)

[Lucy Kay]

Are you comparing repeating the Mann-Whitney test again and again for all possible pairs?

Correct, that is what I am using the Dunn test for. So I am comparing 9 groups in total. Since I am comparing multiple groups (although I am just comparing 2 at a time using the Mann-Whitney test), the p-value obtained from the Mann-Whitney test needs to be multiplied by 9 for a Bonferroni correction (or alternatively, I need to divide the alpha value by 9). I am wondering if, using the Dunn test code in this thread, the p-value is already multiplied by 9 for me?

[Nick Cox]

No; as I understand it,

1. Pairwise Mann-Whitney is fairly dubious procedure, especially as there are procedures like #2.

2. Kruskal-Wallis followed by dunntest is better procedure.

3, There is nothing to fix.

I would give up on linear models unless you are clear about there is nothing that you can deal with using a transformation -- or even better a link function in a generalised linear model.

[Lucy Kay]

I did do #2, so I used a Kruskal Wallis test to see if there is any statistical difference between pairwise comparisons in the 9 groups. Then I did the dunntest to see between which 2 groups the statistical significance lies. But based on my department's Stata notes, I still would need to do a Bonferroni correction for the pairwise Mann-Whitney tests (i.e., multiple the p-value from the pairwise Mann-Whitney tests by the number of groups or divide overall alpha by 9). But since I put a Bonferroni correction into the dunntest code in Stata, I am assuming I don't need to multiple the p-value that the dunntest spits out by 9?