bootstrap estimation

[Varsha Vaishnav]

Hello,

I'm bootstrapping my estimation as follows:

reghdfe x $controls if phase, a(state) vce(cl cluster)
predict x_hat
bootstrap, reps(50): probit y x_hat $controls if phase, vce(cl cluster)
eststo A: margins, dydx(x_hat) post

I have a small doubt. Do I have to drop the variables I'm not utilizing from the dataset to carry out bootstrap estimation? Also, if someone could confirm I'm proceeding the right way with respect to the codes, it would be great.

Thanks

[Maarten Buis]

First question: no you don't need to drop the other variables. second question: that depends on what you want to do, but 50 replications seems low to me.

[Varsha Vaishnav]

Hello Maarten,

I require the estimation for standard errors. If the replications are low, what should I follow then?

[Nick Cox]

Advice to do say 50 replications when it was given hinged on that being a great deal of work. It may not change anyone's mind, but I think most reviewers would expect something more of the order of 1000 here. But we can't tell how much work that would be for your set-up, dataset, model, Stata version, and computer. It also depends on which coefficients are hard to estimate, and so forth.

[Varsha Vaishnav]

Thank you for the detailed response, Nick.

[Carlo Lazzaro]

Varsha:
as an aside to previous helfpul advice, one of the bootstrap holy writings (An Introduction to the Bootstrap | Bradley Efron, R.J. Tibshirani | Ta (taylorfrancis.com)) recommends 25 up to 200 (the upper limit is what would seem acceptable to nasty reviewers) boostrao replications for standard errors.

[Varsha Vaishnav]

Thanks for the reference, Carlo. I'll have a look at it.

[Varsha Vaishnav]

Hello,

When I run these codes:

reghdfe y1 $controls if phase, a(state) vce(cl cluster)
predict y1_hat if e(sample)
bootstrap, reps(50): probit y2 y1_hat $controls1 if phase, vce(cl cluster)

The last one says insufficient observations. What should I do about it? Also, when I run OLS instead of probit, the problem doesn't arise.

[Varsha Vaishnav]

When I use

bootstrap, reps(50): probit y2 y1_hat $controls1 if phase, vce(cl cluster)

or

bootstrap, reps(50): reg y2 y1_hat $controls1 if phase, vce(cl cluster)

, I get the insufficients observations error. But with

bootstrap, reps(50): reghdfe y2 y1_hat $controls2 if phase, a(state) vce(cl cluster)

, I do not. Is it because of the fixed effects? I'm not able to understand. Pls help.

[Carlo Lazzaro]

Varsha:
do you experience convergence issues when you run -probit- outside the -bootstrap- routine?

[Varsha Vaishnav]

Yes, for some regressions, I experienced it.

[Carlo Lazzaro]

Varsha:
this may have a bearing on the subsequent -bootstrap- routine:

 use "C:\Program Files\Stata18\ado\base\a\auto.dta"
(1978 automobile data)

. probit foreign i.rep78

note: 1.rep78 != 0 predicts failure perfectly;
      1.rep78 omitted and 2 obs not used.

note: 2.rep78 != 0 predicts failure perfectly;
      2.rep78 omitted and 8 obs not used.

note: 5.rep78 omitted because of collinearity.
Iteration 0:  Log likelihood = -38.411464  
Iteration 1:  Log likelihood = -27.498115  
Iteration 2:  Log likelihood =   -27.4447  
Iteration 3:  Log likelihood = -27.444671  
Iteration 4:  Log likelihood = -27.444671  

Probit regression                                       Number of obs =     59
                                                        LR chi2(2)    =  21.93
                                                        Prob > chi2   = 0.0000
Log likelihood = -27.444671                             Pseudo R2     = 0.2855

------------------------------------------------------------------------------
     foreign | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
       rep78 |
          1  |          0  (empty)
          2  |          0  (empty)
          3  |  -2.190009   .5397743    -4.06   0.000    -3.247948   -1.132071
          4  |  -.9084579    .530301    -1.71   0.087    -1.947829    .1309129
          5  |          0  (omitted)
             |
       _cons |   .9084579   .4404005     2.06   0.039     .0452887    1.771627
------------------------------------------------------------------------------

. bootstrap(50): probit foreign i.rep78
(running probit on estimation sample)

Bootstrap replications (50): xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx done
x: Error occurred when bootstrap executed probit.
insufficient observations to compute bootstrap standard errors
no results will be saved
r(2000);

.

[Varsha Vaishnav]

I got your point, Carlo. But in the case of OLS, the -bootstrap- runs with -reghdfe- but not with -reg-. Why so?

[Andrew Musau]

Bootstrap replications are random, and there could be differences in the included controls. Also note that unconditional FE probit suffers from the incidental parameters problem (which is severe with low \(T\)). A proper experiment in your case would entail the following (note that reghdfe is from https://github.com/sergiocorreia/reghdfe):

bootstrap, reps(50) seed(09012024): probit y2 y1_hat $controls2 i.state if phase, vce(cl cluster)
bootstrap, reps(50) seed(09012024): regress y2 y1_hat $controls2 i.state if phase, vce(cl cluster)
bootstrap, reps(50) seed(09012024): reghdfe y2 y1_hat $controls2 i.state if phase, noabsorb vce(cl cluster)
bootstrap, reps(50) seed(09012024): reghdfe y2 y1_hat $controls2 if phase, absorb(state) vce(cl cluster)

[Varsha Vaishnav]

Hello Andrew,

The last one works fine. The second and third ones work for some replications. The first one is not working.