Oneway Bonferroni

[Aparna Anand]

Hello,

I am trying to compare variables mean and std deviation across three samples. Is oneway bonferroni test is the best way to do it? If yes, when reporting P value in descriptive statistics table, which p value should be included? ( there are p values for each row column mean comparison and another one for F test) Thanks

[Carlo Lazzaro]

Aparna:
welcome to this forum.
The p-values you mentioned have different meanings:
1) the F-test tests the null that -oneway- does not outperform -mean-;
2) each p-value of the head-to-head comparison provides evidence of rejecting the null that the different groups have the same mean (if p-value<0.05) or the other way round (if p-value>0.05).
3) the absence of evidence of differences in variances is a precondition of -oneway- and can be checked via the outcome of the Bartlett's test (although -oneway- is pretty robust to deviations from variance equality).
As an aside, I would consider -regress- instead of -oneway-, as they're both linear model but there's nothing that -oneway- can do that cannot be done far better by -regress-.

[Carlo Lazzaro]

Just elaborating a bit on the last statement of my previous reply, you can see yourself that -oneway- and -regress- (as expected) reach the same results:

use http://www.stata-press.com/data/r15/apple.dta
. oneway weight treatment, bonferroni

                        Analysis of Variance
    Source              SS         df      MS            F     Prob > F
------------------------------------------------------------------------
Between groups      5295.54433      3   1765.18144     21.46     0.0013
 Within groups      493.591667      6   82.2652778
------------------------------------------------------------------------
    Total             5789.136      9   643.237333

Bartlett's test for equal variances:  chi2(3) =   1.3900  Prob>chi2 = 0.708

             Comparison of Average weight in grams by Fertilizer
                                (Bonferroni)
Row Mean-|
Col Mean |          1          2          3
---------+---------------------------------
       2 |   -59.1667
         |      0.001
         |
       3 |     -33.25    25.9167
         |      0.042      0.122
         |
       4 |      -34.4    24.7667      -1.15
         |      0.036      0.146      1.000

. regress weight i.treatment

      Source |       SS           df       MS      Number of obs   =        10
-------------+----------------------------------   F(3, 6)         =     21.46
       Model |  5295.54433         3  1765.18144   Prob > F        =    0.0013
    Residual |  493.591667         6  82.2652778   R-squared       =    0.9147
-------------+----------------------------------   Adj R-squared   =    0.8721
       Total |    5789.136         9  643.237333   Root MSE        =      9.07

------------------------------------------------------------------------------
      weight |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   treatment |
          2  |  -59.16667   7.405641    -7.99   0.000    -77.28762   -41.04572
          3  |     -33.25   8.279758    -4.02   0.007    -53.50984   -12.99016
          4  |      -34.4   8.279758    -4.15   0.006    -54.65984   -14.14016
             |
       _cons |      111.9   5.236579    21.37   0.000     99.08655    124.7134
------------------------------------------------------------------------------

*Since you have 6 head-to-head comparisons to perform (1 vs 2; 1 vs 3; 1 vs 4; 2 vs 3; 2 vs 4; 3 vs 4) you should multiply by 6 the non-adjusted p-value obtained after -test-*

di 0.000*6
.000
di 0.007*6
.042
di 0.006*6
.036

. test 2.treatment = 3.treatment

 ( 1)  2.treatment - 3.treatment = 0

       F(  1,     6) =    9.80
            Prob > F =    0.0203

. di 0.0203*6
.1218

. test 2.treatment = 4.treatment

 ( 1)  2.treatment - 4.treatment = 0

       F(  1,     6) =    8.95
            Prob > F =    0.0243

. di 0.0243*6
.1458

. test 3.treatment = 4.treatment

 ( 1)  3.treatment - 4.treatment = 0

       F(  1,     6) =    0.02
            Prob > F =    0.9032

*as p-value is capped at 1, there's no scope in multiplying 0.9032 by 6*

.

Moreover, -regress postestimation- offers a suite of test the check possible model misspecifications.