Hello all,
I have a question regarding how to determine the number of independent tests when performing multiple testing corrections.
Suppose my research hypothesis is whether age is related to cognition decline in females, and I have four cognitive outcomes (those cognitive outcomes are also correlated).
In my actual analysis, I tested the association between age and cognition (all four outcomes) in three separate subsample analyses: the full sample, females only, and males only. In this case, should I correct for multiple testing for all 12 tests (3 subsample analyses * four outcomes) or should I correct for multiple testing for only the four outcomes within each subsample analysis (e.g., 4 cognition tests in the female subsample analysis)?
I received conflicting answers when I tried to investigate this question from online sources. I would really appreciate it if anyone could help me answer this question.
I have a question regarding how to determine the number of independent tests when performing multiple testing corrections.
Suppose my research hypothesis is whether age is related to cognition decline in females, and I have four cognitive outcomes (those cognitive outcomes are also correlated).
In my actual analysis, I tested the association between age and cognition (all four outcomes) in three separate subsample analyses: the full sample, females only, and males only. In this case, should I correct for multiple testing for all 12 tests (3 subsample analyses * four outcomes) or should I correct for multiple testing for only the four outcomes within each subsample analysis (e.g., 4 cognition tests in the female subsample analysis)?
I received conflicting answers when I tried to investigate this question from online sources. I would really appreciate it if anyone could help me answer this question.
