Joint hypothesis testing with boottest: boottest var1 var2 OR boottest var1=var2?

[Greg Haugan]

I am trying to run a joint hypothesis test using boottest (owing to few clusters) in Stata 18. I am a little confused about how to apply the proper syntax for my specific case. There are multiple ways to write what I'm trying to do if I were using the test command that would all yield the same result, but writing it in these different ways gives different results with boottest. I think the answer is probably found in the boottest helpfile in the paragraph that begins, "boottest is designed in partial analogy with test," but I'm having a little trouble making sense of everything being described there.

See example below that is analogous to my case:

clear all

*Generate fictional data
set obs 1000
set seed 999
gen year = ceil(runiform(0,10))
gen testscore = ceil(runiform(1,100))
gen panel_unit = ceil(runiform(0,16))
gen treatment = (panel_unit>=8)

*Run model
areg testscore i.treatment##i.year , absorb(panel_unit) vce(cluster panel_unit)


*test gives same result regardless of how this is written
test (1.treatment#2.year=0) (1.treatment#3.year=0) (1.treatment#4.year=0) (1.treatment#5.year=0)
test 1.treatment#2.year =1.treatment#3.year =1.treatment#4.year= 1.treatment#5.year=0

*boottest gives different results depending how it is written
boottest (1.treatment#2.year=0) (1.treatment#3.year=0) (1.treatment#4.year=0) (1.treatment#5.year=0) , cluster(panel_unit) nograph seed(999)
boottest 1.treatment#2.year =1.treatment#3.year =1.treatment#4.year= 1.treatment#5.year=0 , cluster(panel_unit) nograph seed(999)

Could anyone help me make sense of what the difference in the null hypothesis is for these two different specifications of boottest?

Briefly, my purpose for running this is that years 1-5 are a pre-intervention period and I want to run a joint hypothesis test on these estimates as supportive evidence for parallel trends between treatment and control in the pre-intervention period. In other words, the null hypothesis should be that all coefficients are equal to zero.

[Andrew Musau]

boottest is from SSC (FAQ Advice #12). The command works same as the official test command.

boottest var1=var2

tests the hypothesis that the difference between the coefficients on var1 and var2 is zero. This is different from the joint test var1=0, var2=0, which you get with

boottest var1 var2

[Greg Haugan]

Thanks, Andrew. I guess there's a bit of a typo in the title of the thread. It should read: "(var1=0) (var2=0) OR var1=var2=0". And the main reason I'm asking this is because there seems to be a difference between how boottest handles these two different syntaxes, and how they are handled in test. In fact, test seems to treat them as equivalent, but boottest does not. The code I posted should illustrate this.

[Andrew Musau]

The joint test from #2 can be specified as

boottest 1.treatment#2.year 1.treatment#3.year 1.treatment#4.year 1.treatment#5.year, cluster(panel_unit) nograph seed(999)

or

boottest (1.treatment#2.year=0) (1.treatment#3.year=0) (1.treatment#4.year=0) (1.treatment#5.year=0) , cluster(panel_unit) nograph seed(999)

Res.:

. boottest 1.treatment#2.year 1.treatment#3.year 1.treatment#4.year 1.treatment#5.year, cluster(panel_unit) nograph see
> d(999)

Overriding estimator's cluster/robust settings with cluster(panel_unit)

Wild bootstrap-t, null imposed, 999 replications, Wald test, clustering by panel_unit, bootstrap clustering by panel_un
> it, Rademacher weights:
  1.treatment#2.year 1.treatment#3.year 1.treatment#4.year 1.treatment#5.year

                  F(4, 15) =     1.8217
                  Prob > F =     0.3223

. boottest (1.treatment#2.year=0) (1.treatment#3.year=0) (1.treatment#4.year=0) (1.treatment#5.year=0) , cluster(panel_
> unit) nograph seed(999)

Overriding estimator's cluster/robust settings with cluster(panel_unit)

Wild bootstrap-t, null imposed, 999 replications, Wald test, clustering by panel_unit, bootstrap clustering by panel_un
> it, Rademacher weights:
  1.treatment#2.year=0
  1.treatment#3.year=0
  1.treatment#4.year=0
  1.treatment#5.year=0

                  F(4, 15) =     1.8217
                  Prob > F =     0.3223

From #2 again,

boottest var1=var2

tests the hypothesis that the difference between the coefficients on var1 and var2 is zero.

What happens with your code is that boottest will take the first combination (difference) and ignore the rest. Perhaps it should issue an error message instead as this may be confusing to the inattentive user. You could send an email to the author, David Roodman.

boottest 1.treatment#2.year =1.treatment#3.year =1.treatment#4.year= 1.treatment#5.year=0 , cluster(panel_unit) nograph seed(999)
boottest 1.treatment#2.year -1.treatment#3.year =0 , cluster(panel_unit) nograph seed(999)

Res.:

. boottest 1.treatment#2.year =1.treatment#3.year =1.treatment#4.year= 1.treatment#5.year=0 , cluster(panel_unit) nogra
> ph seed(999)

Overriding estimator's cluster/robust settings with cluster(panel_unit)

Wild bootstrap-t, null imposed, 999 replications, Wald test, clustering by panel_unit, bootstrap clustering by panel_un
> it, Rademacher weights:
  1.treatment#2.year =1.treatment#3.year =1.treatment#4.year= 1.treatment#5.year=0

                           t(15) =     0.7493
                        Prob>|t| =     0.4755

95% confidence set for null hypothesis expression: [−11.42, 23.82]

. boottest 1.treatment#2.year -1.treatment#3.year =0 , cluster(panel_unit) nograph seed(999)

Overriding estimator's cluster/robust settings with cluster(panel_unit)

Wild bootstrap-t, null imposed, 999 replications, Wald test, clustering by panel_unit, bootstrap clustering by panel_un
> it, Rademacher weights:
  1.treatment#2.year -1.treatment#3.year =0

                           t(15) =     0.7493
                        Prob>|t| =     0.4755

95% confidence set for null hypothesis expression: [−11.42, 23.82]

[Greg Haugan]

Thank you! This is exactly what I was missing. You're right - for the specification:

 
 test 1.treatment#2.year =1.treatment#3.year =1.treatment#4.year= 1.treatment#5.year=0 boottest 1.treatment#2.year =1.treatment#3.year =1.treatment#4.year= 1.treatment#5.year=0 , cluster(panel_unit) nograph seed(999) 

boottest is not testing the same hypothesis as test. Where boottest testing a single hypothesis, test is testing four.