why different outcomes using -xtreg and -reghdfe

[Eva Gao]

Hi,

My research is studying the relationship between certain policy A which is implemented in some states in the U.S. and cash. The method I use is staggered DiD. The panel data covers all the companies and last for past 20 years. I run the regression adding industry and year FE, using -xtreg and -reghdfe, but the outcomes are totally different. Anyone please tell me why?

. xtreg cash1_w treat_post_lag1 treat_lag1 $control i.sic2 i.fyear

Random-effects GLS regression Number of obs = 93,372
Group variable: gvkey_n Number of groups = 10,810

R-squared: Obs per group:
Within = 0.0415 min = 1
Between = 0.3557 avg = 8.6
Overall = 0.3191 max = 23

Wald chi2(107) = .
corr(u_i, X) = 0 (assumed) Prob > chi2 = .


cash1_w Coefficient Std. err. z P>z [95% conf. interval]

treat_post_lag1 -.0063499 .0020443 -3.11 0.002 -.0103566 -.0023431
treat_lag1 .0495724 .0032204 15.39 0.000 .0432606 .0558842


. reghdfe cash1_w treat_post_lag1 treat_lag1 $control, absorb(fyear sic2)
(MWFE estimator converged in 5 iterations)

HDFE Linear regression Number of obs = 93,372
Absorbing 2 HDFE groups F( 14, 93264) = 1380.62
Prob > F = 0.0000
R-squared = 0.3598
Adj R-squared = 0.3591
Within R-sq. = 0.1717
Root MSE = 0.1833


cash1_w Coefficient Std. err. t P>t [95% conf. interval]

treat_lag1 .0459218 .0015755 29.15 0.000 .0428338 .0490099
treat_post_lag1 .0222396 .0022449 9.91 0.000 .0178397 .0266395

My data sample is as follows:

* Example generated by -dataex-. For more info, type help dataex
clear
input float(cash1_w treat_post_lag1) byte sic2 long gvkey_n double fyear
 .011353784 . 50 1004 1998
.0016747684 0 50 1004 1999
  .01967503 0 50 1004 2000
  .04860891 0 50 1004 2001
  .04246011 0 50 1004 2002
  .05781822 0 50 1004 2003
  .06874616 0 50 1004 2004
  .12437233 0 50 1004 2005
    .078039 0 50 1004 2006
  .08255079 0 50 1004 2007
  .08167267 0 50 1004 2008
   .0528766 0 50 1004 2009
  .03371021 0 50 1004 2010
  .03084276 0 50 1004 2011
  .03523796 0 50 1004 2012
  .04055467 0 50 1004 2013
  .03610561 0 50 1004 2014
 .021635115 0 50 1004 2015
 .006847949 0 50 1004 2016
 .027284056 0 50 1004 2017
 .027089376 0 50 1004 2018
   .2042809 0 50 1004 2019
  .03909853 0 50 1004 2020
  .03742296 0 50 1004 2021
   .1757913 . 36 1013 1999
  .34106535 0 36 1013 2000
  .16874024 0 36 1013 2001
   .2441881 0 36 1013 2002
  .57575756 0 36 1013 2003
   .3512359 0 36 1013 2004
  .29016286 0 36 1013 2005
    .333685 0 36 1013 2006
   .3296691 0 36 1013 2007
    .328735 0 36 1013 2008
   .3985561 0 36 1013 2009
   .4726348 0 36 1013 2010
 .017803518 . 38 1021 1999
  .03738801 0 38 1021 2000
 .022929937 0 38 1021 2001
  .06573249 0 38 1021 2002
   .1088683 0 38 1021 2003
  .05316253 0 38 1021 2004
  .04059855 0 38 1021 2005
  .36350325 0 38 1021 2006
 .033933237 0 38 1021 2007
  .03826924 0 38 1021 2008
 .015216338 . 28 1034 1999
  .04528652 0 28 1034 2000
 .006231778 0 28 1034 2001
  .01039303 0 28 1034 2002
  .02516799 0 28 1034 2003
  .05250514 0 28 1034 2004
   .4928042 0 28 1034 2005
  .12204297 0 28 1034 2006
   .2350809 0 28 1034 2007
   .0435925 . 34 1036 2000
 .007434944 . 36 1037 1998
  .01726776 0 36 1037 1999
  .05491919 0 36 1037 2000
 .010746068 0 36 1037 2001
 .036436863 . 50 1043 1999
 .073479936 . 45 1045 1999
  .08518674 0 45 1045 2000
  .09110563 0 45 1045 2001
  .09029637 0 45 1045 2002
  .10681896 0 45 1045 2003
  .11840962 0 45 1045 2004
  .14660113 0 45 1045 2005
  .17783496 0 45 1045 2006
   .1737076 0 45 1045 2007
  .14164846 0 45 1045 2008
  .19101344 0 45 1045 2009
  .19714604 0 45 1045 2010
   .1987169 0 45 1045 2011
   .2017014 0 45 1045 2012
   .2432944 0 45 1045 2013
  .18452857 0 45 1045 2014
  .14352989 0 45 1045 2015
  .13646293 0 45 1045 2016
  .10475523 0 45 1045 2017
  .08111588 0 45 1045 2018
 .066405535 0 45 1045 2019
   .1205167 0 45 1045 2020
  .20191975 0 45 1045 2021
 .001881357 . 49 1048 1999
.0005830904 0 49 1048 2000
          0 0 49 1048 2001
          0 0 49 1048 2002
 .010836584 0 49 1048 2003
          0 0 49 1048 2004
          0 0 49 1048 2005
          0 0 49 1048 2006
  .07491238 . 35 1050 1999
 .029805353 0 35 1050 2000
.0009994343 0 35 1050 2001
.0041562226 0 35 1050 2002
 .012854158 0 35 1050 2003
 .007803688 0 35 1050 2004
 .007226107 0 35 1050 2005
 .007042476 0 35 1050 2006
end

Thanks,
Eva

[Carlo Lazzaro]

Eva:
your first -xtreg- code calls the -re- estimator (the default setup of -xtreg-), whereas the community-contributed module -reghdfe- uses the -fe- estimator.
Try:

xtreg cash1_w treat_post_lag1 treat_lag1 $control i.sic2 i.fyear, fe

instead and see what Stata gives you back.

[Eva Gao]

Hi Carlo,

Thanks for your reply! The outcome shows like this: (But I am afraid this regression will not consider industry FE. I can see this is totally similar to regression adding firm and year FE, since sic are all omitted in the regression. )

Fixed-effects (within) regression Number of obs = 107,332
Group variable: gvkey_n Number of groups = 12,759

R-squared: Obs per group:
Within = 0.0448 min = 1
Between = 0.1425 avg = 8.4
Overall = 0.1611 max = 23

F(36,94537) = 123.16
corr(u_i, Xb) = 0.0872 Prob > F = 0.0000


cash1_w Coefficient Std. err. t P>t [95% conf. interval]

treat_post_lag1 -.0072495 .0019071 -3.80 0.000 -.0109874 -.0035117
treat_lag1 .0162162 .0029655 5.47 0.000 .0104038 .0220286

Best regards,
Shangyi

[Carlo Lazzaro]

Eva:
you could go:

 
 xtreg cash1_w treat_post_lag1 treat_lag1 $control i.sic2 i.fyear, fe robust

If -i.industry- is a time-invariant predictor, it will be wiped out by the -fe- machinery.

[Eva Gao]

Hi Carlo,

Thanks for your suggestion!

You are right that -i.industry- is a time-invariant predictor, so sic2 are still omitted when adding -fe robust-. And the outcome is still significantly negative, which is inconsistent with using -reghdfe-. 😭

Fixed-effects (within) regression Number of obs = 103,377
Group variable: gvkey_n Number of groups = 11,844

R-squared: Obs per group:
Within = 0.0065 min = 1
Between = 0.0014 avg = 8.7
Overall = 0.0011 max = 23

F(24,11843) = 24.24
corr(u_i, Xb) = -0.0890 Prob > F = 0.0000

(Std. err. adjusted for 11,844 clusters in gvkey_n)

Robust
cash1_w Coefficient std. err. t P>t [95% conf. interval]

treat_post_lag1 -.0093356 .0034799 -2.68 0.007 -.0161568 -.0025144
treat_lag1 -.0023647 .012965 -0.18 0.855 -.0277783 .0230489

Best regards,
Shangyi

[Carlo Lazzaro]

Eva:
your within R-sq is really low.
Check if your model is correctly specified.