Dear Statalist Members
I’ve been asked to continue with the analysis that somebody started and left. The study compares the outcomes (11 outcome measures) between subsequent years. (Below is how their table on descriptive stat (%) looks like)
She compared each outcome between
I’ve been asked to correct for alpha inflation (familywise error rate) for the 33 tests (3 between-year tests for each of the 11 outcomes). I know about the Bonferroni method but that’s too conservative. So I’ve done some reading on this topic to find a better method and learned that there are several methods: Holm’s, Hochberg, & FDR. How do I do any of those in Stata? Moreover, the study team cannot find the cleaned dataset since the data analyst left, so I’m wondering if it is possible to do any of these (Holm’s, Hocheberg, FDR) just from the table on percentages as shown above.
My other question is: Some researchers suggest (Streiner, 2015, Am J Clin Nutr) that when outcomes are set a priori, it is not necessary to correct for multiplicity. Moreover, what vars constitute a “family” of tests may be subjective (Hancock et.al. 1996, Review of Educational research). So do I need to adjust alpha for 11 outcomes (i.e., can I make an argument that we don’t think that 11 outcomes constitute a “family”)? or could I just use alpha correction for the between year tests? For example, if using Bonferroni, that would mean diving alpha (0.05 in our case) by 3 instead of 33. I’m not sure, so would greatly appreciate your suggestions.
Thank you very much in advance for your time!
Sincerely
MN
I’ve been asked to continue with the analysis that somebody started and left. The study compares the outcomes (11 outcome measures) between subsequent years. (Below is how their table on descriptive stat (%) looks like)
| outcomes | 2011 | 2012 | 2013 | 2014 |
| y1 | ||||
| y2 | ||||
| y3 | ||||
| y4 | ||||
| y5 | ||||
| y6 | ||||
| y7 | ||||
| y8 | ||||
| y9 | ||||
| y10 | ||||
| y11 |
- 2011 & 2012,
- 2012 & 2013
- 2013 & 2014
I’ve been asked to correct for alpha inflation (familywise error rate) for the 33 tests (3 between-year tests for each of the 11 outcomes). I know about the Bonferroni method but that’s too conservative. So I’ve done some reading on this topic to find a better method and learned that there are several methods: Holm’s, Hochberg, & FDR. How do I do any of those in Stata? Moreover, the study team cannot find the cleaned dataset since the data analyst left, so I’m wondering if it is possible to do any of these (Holm’s, Hocheberg, FDR) just from the table on percentages as shown above.
My other question is: Some researchers suggest (Streiner, 2015, Am J Clin Nutr) that when outcomes are set a priori, it is not necessary to correct for multiplicity. Moreover, what vars constitute a “family” of tests may be subjective (Hancock et.al. 1996, Review of Educational research). So do I need to adjust alpha for 11 outcomes (i.e., can I make an argument that we don’t think that 11 outcomes constitute a “family”)? or could I just use alpha correction for the between year tests? For example, if using Bonferroni, that would mean diving alpha (0.05 in our case) by 3 instead of 33. I’m not sure, so would greatly appreciate your suggestions.
Thank you very much in advance for your time!
Sincerely
MN
