-boottest- support for fixed effect models

[David Roodman]

The post-estimation testing command -boottest- now works with OLS, 2SLS, and LIML models with one set of fixed effects, as can be run with areg, xtivreg2, reghdfe, xtreg, and xtivreg (the last two with the "fe" option).

[Michael Schleicher]

Dear David,
thanks, your boottest command is very useful.
I am currently running a set of DiD regressions, where I want to apply fixed effects at the lowest possible level (here: enumeration section [se]). I cluster at the district level where the variation in policy reform comes from. reform_t is the DID estimate I am interested in. I am using Stata/MP 14.2.

Boottest just works fine when I throw in dummy variables for all enumerations sections:
reg depvar reform_t t i.se , vce(cluster cnt_d)
boottest reform_t=0.27

Wild bootstrap (Wu 1986), null imposed, 1000 replications, Wald test:

itt = 0.27

quasi-F( 1, 23) = 0.0001
Prob > F = 0.9870

However, it tells me "Doesn't work after xtreg" when I want to run boottest after xtreg:
xtset se t
xtreg depvar reform_t t , fe vce(cluster cnt_d)
boottest reform_t = 0.27
Doesn't work after xtreg.
r(198);

Do you have an idea why I get this error message and how I can fix it?
Many thanks in advance,
Michael

[David Roodman]

Yeah sorry, I think I accidentally sent Kit Baum old boottest files for posting on SSC. The correct ones should be up soon, and should be obtainable via "ssc install boottest, replace".

[Philip Gigliotti]

Yeah sorry, I think I accidentally sent Kit Baum old boottest files for posting on SSC. The correct ones should be up soon, and should be obtainable via "ssc install boottest, replace".

Hi David, I'm trying to run a wild bootstrap after xtreg. I'm running the most recent version of boot test, but it seems like it won't work. Can you post example code for running after

xtreg y x i.year, fe vce(cluster Panel)

[David Roodman]

I was just working on FE support this weekend. The latest is on GitHub. Try:

net from https://raw.github.com/droodman/boottest/v2.1.9
net install boottest, replace

[Philip Gigliotti]

I was just working on FE support this weekend. The latest is on GitHub. Try:

net from https://raw.github.com/droodman/boottest/v2.1.9
net install boottest, replace

Thanks Dave. I have installed the latest version. Possibly I just don't understand the syntax. I just posted a topic with data sample and regression and boot test code. Perhaps you could take a look and show me what I'm doing wrong . I would really appreciate it

[Philip Gigliotti]

I was just working on FE support this weekend. The latest is on GitHub. Try:

net from https://raw.github.com/droodman/boottest/v2.1.9
net install boottest, replace

It works, thanks!

[Sonal Agarwal]

Dear David,

I and a couple of others are facing an issue with the boottest command. We are using Stata/MP 14.1 and trying to run the following commands:

eststo clear

global controls CH_F##CH_BirthOrd CH_Sib M_Age M_Ed M_Wt4Ht HH_Mus HH_Hin HH_SCST HH_Urban HH_Mem HH_Wealth HH_Fam

eststo xi: reghdfe NN_D Treat_E##FB_f##CH_F Treat_P Treat_S $controls [pw=perweight] if kidbirthy>1985, a(i.district##c.kidbirthyr) cluster(stateid)
boottest 1.Treat_E#1.FB_f#1.CH_F, reps (9999) nograph noci bootcluster (stateid)
return list
estadd scalar p_boot = r(p)

These commands return the following error "factor variables and time-series operators are not allowed" when trying to run boottest. But when someone else tried on their computer, it worked for them. They use the same state version and ran the same commands as these. We're really unsure what the issue might be? There has been no post resolving this kind of an error. It would be really helpful if you could help us out.

Thanks,

Sonal

[David Roodman]

boottest definitely cannot handle "a(i.district##c.kidbirthyr)". So then the question is why you don't always get the error message. I will investigate.

[David Roodman]

I've done some tests in Stata 13.1 and 16.1. I get that error message in both versions, and the error message is appropriate.

[Sonal Agarwal]

Dear David,

If I try to run the command excluding all the absorb variables and using reg instead of reghdfe, I still get error r(111). Though I no longer get an error for factor and time variables. It is quite possible that my other colleague used a slightly different specification than mine, which they're looking into.

If you can figure out a way of using the boottest command with the aforementioned absorb specification, it would be really helpful.

Thanks,

Sonal

[Maxence Morlet]

Dear David Roodman,

Thank you very much for the helpful boottest command!

I just wanted to know;

I ran the following command:

boottest regressor =0, reps(9999) boottype(wild) seed(123)

after a reghdfe estimation with multiple interacted fixed effects (e.g. occupation times time, industry times time, etc.)

and got the following error message:

"Doesn't work after reghdfe with more than one set of absorbed fixed effects or with absorbed interaction
> terms.
r(198);
"

Is there an econometric reason behind this?

[David Roodman]

The point of the reghdfe command is that time-efficient estimation of models with multiple sets of fixed effects is a nontrivial problem. For boottest to work in this context, it would need to extend such methods to 10,000 regressions. That's a big challenge.

[Maxence Morlet]

I see, thanks for explaining it!

Out of interest, and perhaps other Statalisters may be interested in your response, I would be very keen to hear your expert opinion on the following (I've read your very informative 2018 paper "Fast and Wild: Bootstrap Inference in Stata using boottest"):

I currently have a panel daatset with the sources of variation described above. I have tried using so-called CV1 standard errors (the default standard errors obtained on Stata when the user specifies cluster(clustvar)), however as noted in MacKinnon et al. (2022), these standard errors may over-reject severely when clusters are very heterogenous.

I wanted to conduct robustness checks using the wild cluster restricted bootstrap with the boottest command, however got the above error.

I will try using the community-contributed command summclust to obtain so-called CV3 standard errors, however I am having difficulties finding the right specification to run this command without errors.

Do you have any recommendations on how I can achieve conservative inference? My number of clusters is very large, however each cluster is very heterogeneous in terms of number of observations, leverage, etc.

[David Roodman]

Is it practical for you to move all but the largest set of fixed effects into your regressor list? I think boottest should be able to handle something like "reghdfe ..., a(b#c)" because that is just one set of absorbed fixed effects. boottest does not support the CV3 but it can now jackknife the residuals used in the bootstrap, on the way to computing the CV1 in each bootstrap replication, which MacKinnon et al. find performs well. They call that the WCR31, though apparently its name is now changing to WCR-S.