flipping sign when adding different fixed effects

[Eva Gao]

Hi,

My research is studying the relationship between certain policy A which is implemented in some states in the U.S. and companies' financial decision B. The method I use is staggered DiD. The panel data covers all the companies and last for past 20 years. And I assume the relationship is positive.

when I add fixed effect into regression, most of the outcomes are significantly positive, including only year FE, only firm FE, only industry FE, and year*industry FE. However, when I add firm*year FE, the outcome is significantly negative. I am really confused and is there anyone can kindly help me out? I really appreciate any suggestion here.

Please see the code and regression below. Please note that treat and post are omitted because of collinearity.
reghdfe B post*treat $control, absorb( gvkey fyear )

	B1	B2	B3	B4	B5
treat_post~1	-0.00739**	0.00748	-0.0756***	-0.0837***	-0.00602***
	(-3.12)	-0.22	(-4.14)	(-3.96)	(-3.39)
_cons	0.337***	2.321***	-1.796***	-1.144***	0.266***
	-84.62	-40.22	(-57.89)	(-31.80)	-89.12
N	68806	68760	68206	68159	68806

Best regards,
Eva

[Eva Gao]

can someone help me please?

[Carlo Lazzaro]

Eva:
without an excerpt/example of your data thatb you can post via -dataex- it is difficul to say.
Maybe the correlatioin between industry and firms contributes to explain the issue you're experiencing.

[Maxence Morlet]

When adding firm-year FE, you are comparing observations within firm-year cells only, never across. The variation used by your estimator changes.

[Eva Gao]

Eva:
without an excerpt/example of your data thatb you can post via -dataex- it is difficul to say.
Maybe the correlatioin between industry and firms contributes to explain the issue you're experiencing.

Hi Carlo,

Thanks for your kind help. The correlation coefficient of gvkey and sic is 0.04, which is very low, if you mean that.

Please see the table below.

. dataex y treat_post_lag1 sic2 gvkey_n fyear

* Example generated by -dataex-. For more info, type help dataex
clear
input float(cash1_w treat_post_lag1) byte sic2 long gvkey_n double fyear
 .011353784 . 50 1004 1998
.0016747684 0 50 1004 1999
  .01967503 0 50 1004 2000
  .04860891 0 50 1004 2001
  .04246011 0 50 1004 2002
  .05781822 0 50 1004 2003
  .06874616 0 50 1004 2004
  .12437233 0 50 1004 2005
    .078039 0 50 1004 2006
  .08255079 0 50 1004 2007
  .08167267 0 50 1004 2008
   .0528766 0 50 1004 2009
  .03371021 0 50 1004 2010
  .03084276 0 50 1004 2011
  .03523796 0 50 1004 2012
  .04055467 0 50 1004 2013
  .03610561 0 50 1004 2014
 .021635115 0 50 1004 2015
 .006847949 0 50 1004 2016
 .027284056 0 50 1004 2017
 .027089376 0 50 1004 2018
   .2042809 0 50 1004 2019
  .03909853 0 50 1004 2020
  .03742296 0 50 1004 2021
   .1757913 . 36 1013 1999
  .34106535 0 36 1013 2000
  .16874024 0 36 1013 2001
   .2441881 0 36 1013 2002
  .57575756 0 36 1013 2003
   .3512359 0 36 1013 2004
  .29016286 0 36 1013 2005
    .333685 0 36 1013 2006
   .3296691 0 36 1013 2007
    .328735 0 36 1013 2008
   .3985561 0 36 1013 2009
   .4726348 0 36 1013 2010
 .017803518 . 38 1021 1999
  .03738801 0 38 1021 2000
 .022929937 0 38 1021 2001
  .06573249 0 38 1021 2002
   .1088683 0 38 1021 2003
  .05316253 0 38 1021 2004
  .04059855 0 38 1021 2005
  .36350325 0 38 1021 2006
 .033933237 0 38 1021 2007
  .03826924 0 38 1021 2008
 .015216338 . 28 1034 1999
  .04528652 0 28 1034 2000
 .006231778 0 28 1034 2001
  .01039303 0 28 1034 2002
  .02516799 0 28 1034 2003
  .05250514 0 28 1034 2004
   .4928042 0 28 1034 2005
  .12204297 0 28 1034 2006
   .2350809 0 28 1034 2007
   .0435925 . 34 1036 2000
 .007434944 . 36 1037 1998
  .01726776 0 36 1037 1999
  .05491919 0 36 1037 2000
 .010746068 0 36 1037 2001
 .036436863 . 50 1043 1999
 .073479936 . 45 1045 1999
  .08518674 0 45 1045 2000
  .09110563 0 45 1045 2001
  .09029637 0 45 1045 2002
  .10681896 0 45 1045 2003
  .11840962 0 45 1045 2004
  .14660113 0 45 1045 2005
  .17783496 0 45 1045 2006
   .1737076 0 45 1045 2007
  .14164846 0 45 1045 2008
  .19101344 0 45 1045 2009
  .19714604 0 45 1045 2010
   .1987169 0 45 1045 2011
   .2017014 0 45 1045 2012
   .2432944 0 45 1045 2013
  .18452857 0 45 1045 2014
  .14352989 0 45 1045 2015
  .13646293 0 45 1045 2016
  .10475523 0 45 1045 2017
  .08111588 0 45 1045 2018
 .066405535 0 45 1045 2019
   .1205167 0 45 1045 2020
  .20191975 0 45 1045 2021
 .001881357 . 49 1048 1999
.0005830904 0 49 1048 2000
          0 0 49 1048 2001
          0 0 49 1048 2002
 .010836584 0 49 1048 2003
          0 0 49 1048 2004
          0 0 49 1048 2005
          0 0 49 1048 2006
  .07491238 . 35 1050 1999
 .029805353 0 35 1050 2000
.0009994343 0 35 1050 2001
.0041562226 0 35 1050 2002
 .012854158 0 35 1050 2003
 .007803688 0 35 1050 2004
 .007226107 0 35 1050 2005
 .007042476 0 35 1050 2006
end

Best regards,
Eva

[Eva Gao]

When adding firm-year FE, you are comparing observations within firm-year cells only, never across. The variation used by your estimator changes.

Thanks for your kind reply! I am not sure I understand the meaning of within and across. if that means when adding firm-year EF, the effect of individual-level characteristics that vary over time but are constant across individuals will not be considered? and the effect of individual-level characteristics that are constant over time and vary across individuals will not be considered either?

Best regards,
Eva

[Maxence Morlet]

Suppose a regression with no fixed effects at all, and then a regression with firm-year FE. In the first regression, you will be comparing Walmart to ExxonMobil and in different years. With firm-year fixed-effects, you will be comparing observations belonging to Walmart in 2013 to other observations belonging to Walmart in 2013.

That's just an example to illustrate, let me know if it's still unclear

[Eva Gao]

Suppose a regression with no fixed effects at all, and then a regression with firm-year FE. In the first regression, you will be comparing Walmart to ExxonMobil and in different years. With firm-year fixed-effects, you will be comparing observations belonging to Walmart in 2013 to other observations belonging to Walmart in 2013.

That's just an example to illustrate, let me know if it's still unclear

Thank you Maxence, it is clear to me now. So, it seems like there are some problems with my data? but not how I analyze it?

[Carlo Lazzaro]

Eva:

. xtset gvkey_n fyear

Panel variable: gvkey_n (unbalanced)
 Time variable: fyear, 1998 to 2021
         Delta: 1 unit

. xtreg cash1_w i.treat_post_lag1 i.fyear, vce(cluster gvkey_n) fe
note: 0.treat_post_lag1 omitted because of collinearity.

Fixed-effects (within) regression               Number of obs     =         90
Group variable: gvkey_n                         Number of groups  =          8

R-squared:                                      Obs per group:
     Within  = 0.2338                                         min =          3
     Between = 0.1975                                         avg =       11.3
     Overall = 0.1342                                         max =         23

                                                F(7,7)            =          .
corr(u_i, Xb) = 0.0876                          Prob > F          =          .

                                     (Std. err. adjusted for 8 clusters in gvkey_n)
-----------------------------------------------------------------------------------
                  |               Robust
          cash1_w | Coefficient  std. err.      t    P>|t|     [95% conf. interval]
------------------+----------------------------------------------------------------
0.treat_post_lag1 |          0  (omitted)
                  |
            fyear |
            2000  |   .0240054   .0123645     1.94   0.093    -.0052321    .0532429
            2001  |   -.009063    .020919    -0.43   0.678    -.0585285    .0404025
            2002  |   .0082891   .0155965     0.53   0.612    -.0285907     .045169
            2003  |   .0712742   .0460432     1.55   0.166    -.0376006     .180149
            2004  |   .0360944   .0114103     3.16   0.016     .0091134    .0630754
            2005  |   .1003661   .0667999     1.50   0.177    -.0575904    .2583227
            2006  |   .0975636   .0473824     2.06   0.078    -.0144779    .2096051
            2007  |   .0746978   .0326332     2.29   0.056    -.0024675     .151863
            2008  |   .0455396   .0165127     2.76   0.028     .0064932    .0845859
            2009  |   .0897826   .0256745     3.50   0.010      .029072    .1504931
            2010  |   .1101309   .0505656     2.18   0.066    -.0094377    .2296994
            2011  |   .0730374   .0422813     1.73   0.128     -.026942    .1730168
            2012  |   .0767273   .0416812     1.84   0.108    -.0218332    .1752877
            2013  |   .1001821   .0572096     1.75   0.123    -.0350971    .2354613
            2014  |   .0685747    .034053     2.01   0.084    -.0119479    .1490972
            2015  |   .0408401   .0231181     1.77   0.121    -.0138255    .0955057
            2016  |    .029913   .0262455     1.14   0.292    -.0321477    .0919737
            2017  |   .0242772    .009257     2.62   0.034      .002388    .0461665
            2018  |   .0123602   .0122803     1.01   0.348     -.016678    .0413984
            2019  |   .0936008   .0923359     1.01   0.344    -.1247389    .3119405
            2020  |   .0380652   .0097594     3.90   0.006     .0149879    .0611425
            2021  |   .0779289   .0408454     1.91   0.098    -.0186551     .174513
                  |
            _cons |   .0601426   .0082923     7.25   0.000     .0405344    .0797509
------------------+----------------------------------------------------------------
          sigma_u |  .10675744
          sigma_e |  .07990625
              rho |  .64093175   (fraction of variance due to u_i)
-----------------------------------------------------------------------------------

.

shows that you have a time-invariant predictor (-0.treat_post_lag1-) taht, as expected. cannot be estimated via -fe-, as this estimator focuses on the within-panel variation (as Maxence helpfully highlighted).
In addition, with 8 panels only, my cluster-robust standard errors are clearly misleading (you should have at least 30 panels to trust cluster-robust standard errors; see https://cameron.econ.ucdavis.edu/res...5_February.pdf)

[Andrew Musau]

Suppose a regression with no fixed effects at all, and then a regression with firm-year FE. In the first regression, you will be comparing Walmart to ExxonMobil and in different years. With firm-year fixed-effects, you will be comparing observations belonging to Walmart in 2013 to other observations belonging to Walmart in 2013.

You have to be careful with this kind of reasoning. The within-estimator is one way to estimate the fixed effects (FE) model. But you could also estimate the model by adding an indicator for each unit and subsequently running OLS (the so-called least squares dummy variables [LSDV] estimator). Ultimately, this kind of reasoning is what leads many to incorrectly conclude that "I am interested in variation within units" = FE and "I am interested in variation across units" = Between effects / Random effects. The correct way to think of FE is a way to control for unobserved heterogeneity across units. See post #11 in https://www.statalist.org/forums/for...nel-regression.

[Eva Gao]

Eva:

. xtset gvkey_n fyear

Panel variable: gvkey_n (unbalanced)
Time variable: fyear, 1998 to 2021
Delta: 1 unit

. xtreg cash1_w i.treat_post_lag1 i.fyear, vce(cluster gvkey_n) fe
note: 0.treat_post_lag1 omitted because of collinearity.

Fixed-effects (within) regression Number of obs = 90
Group variable: gvkey_n Number of groups = 8

R-squared: Obs per group:
Within = 0.2338 min = 3
Between = 0.1975 avg = 11.3
Overall = 0.1342 max = 23

F(7,7) = .
corr(u_i, Xb) = 0.0876 Prob > F = .

(Std. err. adjusted for 8 clusters in gvkey_n)
-----------------------------------------------------------------------------------
| Robust
cash1_w | Coefficient std. err. t P>|t| [95% conf. interval]
------------------+----------------------------------------------------------------
0.treat_post_lag1 | 0 (omitted)
|
fyear |
2000 | .0240054 .0123645 1.94 0.093 -.0052321 .0532429
2001 | -.009063 .020919 -0.43 0.678 -.0585285 .0404025
2002 | .0082891 .0155965 0.53 0.612 -.0285907 .045169
2003 | .0712742 .0460432 1.55 0.166 -.0376006 .180149
2004 | .0360944 .0114103 3.16 0.016 .0091134 .0630754
2005 | .1003661 .0667999 1.50 0.177 -.0575904 .2583227
2006 | .0975636 .0473824 2.06 0.078 -.0144779 .2096051
2007 | .0746978 .0326332 2.29 0.056 -.0024675 .151863
2008 | .0455396 .0165127 2.76 0.028 .0064932 .0845859
2009 | .0897826 .0256745 3.50 0.010 .029072 .1504931
2010 | .1101309 .0505656 2.18 0.066 -.0094377 .2296994
2011 | .0730374 .0422813 1.73 0.128 -.026942 .1730168
2012 | .0767273 .0416812 1.84 0.108 -.0218332 .1752877
2013 | .1001821 .0572096 1.75 0.123 -.0350971 .2354613
2014 | .0685747 .034053 2.01 0.084 -.0119479 .1490972
2015 | .0408401 .0231181 1.77 0.121 -.0138255 .0955057
2016 | .029913 .0262455 1.14 0.292 -.0321477 .0919737
2017 | .0242772 .009257 2.62 0.034 .002388 .0461665
2018 | .0123602 .0122803 1.01 0.348 -.016678 .0413984
2019 | .0936008 .0923359 1.01 0.344 -.1247389 .3119405
2020 | .0380652 .0097594 3.90 0.006 .0149879 .0611425
2021 | .0779289 .0408454 1.91 0.098 -.0186551 .174513
|
_cons | .0601426 .0082923 7.25 0.000 .0405344 .0797509
------------------+----------------------------------------------------------------
sigma_u | .10675744
sigma_e | .07990625
rho | .64093175 (fraction of variance due to u_i)
-----------------------------------------------------------------------------------

.

shows that you have a time-invariant predictor (-0.treat_post_lag1-) taht, as expected. cannot be estimated via -fe-, as this estimator focuses on the within-panel variation (as Maxence helpfully highlighted).
In addition, with 8 panels only, my cluster-robust standard errors are clearly misleading (you should have at least 30 panels to trust cluster-robust standard errors; see https://cameron.econ.ucdavis.edu/res...5_February.pdf)

Thank you Carlo! It's quite clear now!