Dear STATA users,
To conduct a robustness check, I used multiple hypothesis testing since I am testing many outcomes so I want to avoid Type 1 error.
Just a context, my data is a data from a previous randomized controlled trial (RCT).
I used the rwolf command for the Multiple Hypothesis Testing and the result is as follows:
------------------------------------------------------------------------------
Outcome Variable | Model p-value Resample p-value Romano-Wolf p-value
--------------------+---------------------------------------------------------
dm_educ | 0.1561 0.1988 0.3546
dm_health | 0.2753 0.2947 0.4595
dm_disiplin | 0.0699 0.1299 0.2218
dm_kb | 0.3488 0.4086 0.4595
------------------------------------------------------------------------------
Uncorrected (model) p-values result in 0 rejected null hypotheses (95% level).
Romano-Wolf adjusted p-values result in 0 rejected null hypotheses (95% level).
I just find it strange that the p-value here even for the corrected model shows no significant relationship between the independent variable and the outcome variable but when I did the independent regression, the result is significant (I used treatreg for the independent regression and ivreg for the MHT), is this why the results differ? Please help me, thanks so much!
To conduct a robustness check, I used multiple hypothesis testing since I am testing many outcomes so I want to avoid Type 1 error.
Just a context, my data is a data from a previous randomized controlled trial (RCT).
I used the rwolf command for the Multiple Hypothesis Testing and the result is as follows:
------------------------------------------------------------------------------
Outcome Variable | Model p-value Resample p-value Romano-Wolf p-value
--------------------+---------------------------------------------------------
dm_educ | 0.1561 0.1988 0.3546
dm_health | 0.2753 0.2947 0.4595
dm_disiplin | 0.0699 0.1299 0.2218
dm_kb | 0.3488 0.4086 0.4595
------------------------------------------------------------------------------
Uncorrected (model) p-values result in 0 rejected null hypotheses (95% level).
Romano-Wolf adjusted p-values result in 0 rejected null hypotheses (95% level).
I just find it strange that the p-value here even for the corrected model shows no significant relationship between the independent variable and the outcome variable but when I did the independent regression, the result is significant (I used treatreg for the independent regression and ivreg for the MHT), is this why the results differ? Please help me, thanks so much!
