Boottest command for multiple hypothesis testing

[Tariku Getaneh]

Dear Statalists,
Hoping for someone to explain: I want to know whether boottest, a user-written command by David for wild bootstrapping, will be useful for the following situation.

I want to estimate the effects of policy intervention (T) on multiple outcomes (about 27); In this case, should I account for multiple hypothesis testing?
I am interested in the effects of the policy (T) and I did this for each outcome.

reg Y1 T (treatment) Xlist
boottest T, madjust(bonferroni)
.
.
.
reg Yn T (treatment) Xlist
boottest T, madjust(bonferroni)

P values are the same with and without madjust option

Can anyone explain how to handle the multiple hypothesis testing when using boottest?

With warm regards,

[Fei Wang]

Tariku, if you'd like to test for the joint significance of the 27 coefficients of "T", you may use mvreg. Below is a simple example where there are three outcomes and the independent variable is "foreign".

sysuse auto, clear
mvreg price weight mpg = foreign
test foreign

The testing results are

 ( 1)  [price]foreign = 0
 ( 2)  [weight]foreign = 0
 ( 3)  [mpg]foreign = 0

       F(  3,    72) =   29.98
            Prob > F =    0.0000

[Tariku Getaneh]

Dear Fei,
Look my model is Difference in Difference and I have to cluster Standard Error (SE), but my clusters are small so I have to use boottest in order to account for a small number of clusters. This command - boottest- corrects SE and gives me adjusted p-values, but I need to adjust for also the multiple hypothesis testing (27 outcomes).

[Fei Wang]

boottest can be used after cmp, sem and gsem. You may combine the 27 equations together with either of the three commands and then implement the multiple hypothesis testing with boottest.

[Tariku Getaneh]

boottest can be used after cmp, sem and gsem. You may combine the 27 equations together with either of the three commands and then implement the multiple hypothesis testing with boottest.

Dear Fei,
Could you please give me an example in Stata syntax? I'm knew to cmp, sem, gsem.
I'm using OLS for each outcomes: reg Y treated_post X

[Fei Wang]

Dear Fei,
Could you please give me an example in Stata syntax? I'm knew to cmp, sem, gsem.
I'm using OLS for each outcomes: reg Y treated_post X

Please refer to #1 of the thread, where cmp is used to combine multiple regressions and boottest is then used for multiple hypothesis testing.

[Lanna Ng]

Dear Statalists,
Hoping for someone to explain: I want to know whether boottest, a user-written command by David for wild bootstrapping, will be useful for the following situation.

I want to estimate the effects of policy intervention (T) on multiple outcomes (about 27); In this case, should I account for multiple hypothesis testing?
I am interested in the effects of the policy (T) and I did this for each outcome.

reg Y1 T (treatment) Xlist
boottest T, madjust(bonferroni)
.
.
.
reg Yn T (treatment) Xlist
boottest T, madjust(bonferroni)

P values are the same with and without madjust option

Can anyone explain how to handle the multiple hypothesis testing when using boottest?

With warm regards,

Dear Tariku,
I would like to see if you have solved your issue or not? I am having the same issues with small sample size (36). However, I am using Romano-Wolf procedure, which is also bootstrap based. Therefore, I am confused how to combine Wild-cluster bootstrap and the multiple hypothesis testing together, or whether it is necessarily to combine both.
Thanks!
Lanna