Different standard errors for -reg- vs -reghdfe- in panel OLS regression with fixed-effect & error clustering

[Stephan Scheffelberg]

Hello,

I noticed that I get (slightly) different standard errors (.0414944 vs. .0412343) depending on whether I use -regress- or -reghdfe- in the results of following panel OLS regressions using both fixed-effect and clustering of errors on a given variable:

regress gdp population i.year_publication, vce(cluster year_publication)
reghdfe gdp population, absorb(year_publication) vce(cluster year_publication)

The error is fairly minimal (ca 0.5%).

My questions are:

1. Is this simply difference due to rounding/approximation errors between both implementations or do both regressions mean something different?
2. Is one implementation “more correct” or can the error be disregarded?

---

Reproducible Example:

/* Install dependencies */

ssc install ftools
ssc install reghdfe


/* Load data */

clear
input str6 continent str7 country str9 source int(year_publication year_data population) float gdp
"Africa" "Angola"  "OECD"      2020 2018 972  52.69
"Africa" "Angola"  "OECD"      2020 2019 986  802.7
"Africa" "Angola"  "OECD"      2020 2020 641 568.74
"Africa" "Angola"  "OECD"      2021 2018 438 168.83
"Africa" "Angola"  "OECD"      2021 2019 958 310.57
"Africa" "Angola"  "OECD"      2021 2020 270 144.02
"Africa" "Angola"  "OECD"      2022 2018 528 359.71
"Africa" "Angola"  "OECD"      2022 2019 974 582.98
"Africa" "Angola"  "OECD"      2022 2020 835 820.49
"Africa" "Angola"  "IMF"       2020 2018 168 148.85
"Africa" "Angola"  "IMF"       2020 2019 460 236.21
"Africa" "Angola"  "IMF"       2020 2020 360 297.15
"Africa" "Angola"  "IMF"       2021 2018 381 249.13
"Africa" "Angola"  "IMF"       2021 2019 648 128.05
"Africa" "Angola"  "IMF"       2021 2020 206 179.05
"Africa" "Angola"  "IMF"       2022 2018 282 150.29
"Africa" "Angola"  "IMF"       2022 2019 125  23.42
"Africa" "Angola"  "IMF"       2022 2020 410 247.35
"Africa" "Angola"  "WorldBank" 2020 2018 553 182.06
"Africa" "Angola"  "WorldBank" 2020 2019 847 698.87
"Africa" "Angola"  "WorldBank" 2020 2020 844 126.61
"Africa" "Angola"  "WorldBank" 2021 2018 307 239.76
"Africa" "Angola"  "WorldBank" 2021 2019 659 510.73
"Africa" "Angola"  "WorldBank" 2021 2020 548 331.89
"Africa" "Angola"  "WorldBank" 2022 2018 448 122.76
"Africa" "Angola"  "WorldBank" 2022 2019 768 761.41
"Africa" "Angola"  "WorldBank" 2022 2020 324 163.57
"Africa" "Benin"   "OECD"      2020 2018 513  196.9
"Africa" "Benin"   "OECD"      2020 2019 590   83.7
"Africa" "Benin"   "OECD"      2020 2020 791 511.09
"Africa" "Benin"   "OECD"      2021 2018 799 474.43
"Africa" "Benin"   "OECD"      2021 2019 455 234.21
"Africa" "Benin"   "OECD"      2021 2020 549 238.83
"Africa" "Benin"   "OECD"      2022 2018 235 229.33
"Africa" "Benin"   "OECD"      2022 2019 347  46.51
"Africa" "Benin"   "OECD"      2022 2020 532 392.13
"Africa" "Benin"   "IMF"       2020 2018 138 137.05
"Africa" "Benin"   "IMF"       2020 2019 978 239.82
"Africa" "Benin"   "IMF"       2020 2020 821  33.41
"Africa" "Benin"   "IMF"       2021 2018 453 291.93
"Africa" "Benin"   "IMF"       2021 2019 526 381.88
"Africa" "Benin"   "IMF"       2021 2020 467 313.57
"Africa" "Benin"   "IMF"       2022 2018 948 555.23
"Africa" "Benin"   "IMF"       2022 2019 323 289.91
"Africa" "Benin"   "IMF"       2022 2020 421  62.35
"Africa" "Benin"   "WorldBank" 2020 2018 983 271.69
"Africa" "Benin"   "WorldBank" 2020 2019 138  23.55
"Africa" "Benin"   "WorldBank" 2020 2020 636 623.65
"Africa" "Benin"   "WorldBank" 2021 2018 653 534.99
"Africa" "Benin"   "WorldBank" 2021 2019 564  368.8
"Africa" "Benin"   "WorldBank" 2021 2020 741 312.02
"Africa" "Benin"   "WorldBank" 2022 2018 328 292.11
"Africa" "Benin"   "WorldBank" 2022 2019 653 429.21
"Africa" "Benin"   "WorldBank" 2022 2020 951 242.73
"Africa" "Chad"    "OECD"      2020 2018 176  95.06
"Africa" "Chad"    "OECD"      2020 2019 783 425.34
"Africa" "Chad"    "OECD"      2020 2020 885  461.6
"Africa" "Chad"    "OECD"      2021 2018 673  15.87
"Africa" "Chad"    "OECD"      2021 2019 131  74.46
"Africa" "Chad"    "OECD"      2021 2020 430  61.58
"Africa" "Chad"    "OECD"      2022 2018 593 211.34
"Africa" "Chad"    "OECD"      2022 2019 647 550.37
"Africa" "Chad"    "OECD"      2022 2020 154 105.65
"Africa" "Chad"    "IMF"       2020 2018 160  32.41
"Africa" "Chad"    "IMF"       2020 2019 654  27.84
"Africa" "Chad"    "IMF"       2020 2020 616 468.92
"Africa" "Chad"    "IMF"       2021 2018 996   22.4
"Africa" "Chad"    "IMF"       2021 2019 126  93.18
"Africa" "Chad"    "IMF"       2021 2020 879 547.87
"Africa" "Chad"    "IMF"       2022 2018 663    520
"Africa" "Chad"    "IMF"       2022 2019 681 544.76
"Africa" "Chad"    "IMF"       2022 2020 101   55.6
"Africa" "Chad"    "WorldBank" 2020 2018 786 757.22
"Africa" "Chad"    "WorldBank" 2020 2019 599 593.69
"Africa" "Chad"    "WorldBank" 2020 2020 641 529.84
"Africa" "Chad"    "WorldBank" 2021 2018 343 287.89
"Africa" "Chad"    "WorldBank" 2021 2019 438 340.83
"Africa" "Chad"    "WorldBank" 2021 2020 762 594.67
"Africa" "Chad"    "WorldBank" 2022 2018 430 128.69
"Africa" "Chad"    "WorldBank" 2022 2019 260 242.59
"Africa" "Chad"    "WorldBank" 2022 2020 607  216.1
"Europe" "Denmark" "OECD"      2020 2018 114  86.75
"Europe" "Denmark" "OECD"      2020 2019 937 373.29
"Europe" "Denmark" "OECD"      2020 2020 866 392.93
"Europe" "Denmark" "OECD"      2021 2018 296  41.04
"Europe" "Denmark" "OECD"      2021 2019 402  32.67
"Europe" "Denmark" "OECD"      2021 2020 306   7.88
"Europe" "Denmark" "OECD"      2022 2018 540 379.51
"Europe" "Denmark" "OECD"      2022 2019 108  26.72
"Europe" "Denmark" "OECD"      2022 2020 752  307.2
"Europe" "Denmark" "IMF"       2020 2018 157  24.24
"Europe" "Denmark" "IMF"       2020 2019 303  79.04
"Europe" "Denmark" "IMF"       2020 2020 286 122.36
"Europe" "Denmark" "IMF"       2021 2018 569  69.32
"Europe" "Denmark" "IMF"       2021 2019 808 642.67
"Europe" "Denmark" "IMF"       2021 2020 157   5.58
"Europe" "Denmark" "IMF"       2022 2018 147 112.21
"Europe" "Denmark" "IMF"       2022 2019 414 311.16
"Europe" "Denmark" "IMF"       2022 2020 774 230.46
"Europe" "Denmark" "WorldBank" 2020 2018 695 350.03
"Europe" "Denmark" "WorldBank" 2020 2019 511 209.84
"Europe" "Denmark" "WorldBank" 2020 2020 181  29.27
"Europe" "Denmark" "WorldBank" 2021 2018 503 176.89
"Europe" "Denmark" "WorldBank" 2021 2019 710 609.02
"Europe" "Denmark" "WorldBank" 2021 2020 264 165.78
"Europe" "Denmark" "WorldBank" 2022 2018 670 638.99
"Europe" "Denmark" "WorldBank" 2022 2019 651  354.6
"Europe" "Denmark" "WorldBank" 2022 2020 632 623.94
"Europe" "Estonia" "OECD"      2020 2018 838 263.67
"Europe" "Estonia" "OECD"      2020 2019 638 533.95
"Europe" "Estonia" "OECD"      2020 2020 898 638.73
"Europe" "Estonia" "OECD"      2021 2018 262  98.16
"Europe" "Estonia" "OECD"      2021 2019 569 552.54
"Europe" "Estonia" "OECD"      2021 2020 868 252.48
"Europe" "Estonia" "OECD"      2022 2018 927 264.65
"Europe" "Estonia" "OECD"      2022 2019 205  150.6
"Europe" "Estonia" "OECD"      2022 2020 828 752.61
"Europe" "Estonia" "IMF"       2020 2018 841 176.31
"Europe" "Estonia" "IMF"       2020 2019 614 230.55
"Europe" "Estonia" "IMF"       2020 2020 500  41.19
"Europe" "Estonia" "IMF"       2021 2018 510 169.68
"Europe" "Estonia" "IMF"       2021 2019 765 401.85
"Europe" "Estonia" "IMF"       2021 2020 751  319.6
"Europe" "Estonia" "IMF"       2022 2018 314  58.81
"Europe" "Estonia" "IMF"       2022 2019 155   2.24
"Europe" "Estonia" "IMF"       2022 2020 734  187.6
"Europe" "Estonia" "WorldBank" 2020 2018 332 160.17
"Europe" "Estonia" "WorldBank" 2020 2019 466 385.33
"Europe" "Estonia" "WorldBank" 2020 2020 487 435.06
"Europe" "Estonia" "WorldBank" 2021 2018 461 249.19
"Europe" "Estonia" "WorldBank" 2021 2019 932 763.38
"Europe" "Estonia" "WorldBank" 2021 2020 650 463.91
"Europe" "Estonia" "WorldBank" 2022 2018 570 549.97
"Europe" "Estonia" "WorldBank" 2022 2019 909  80.48
"Europe" "Estonia" "WorldBank" 2022 2020 523 242.22
"Europe" "Finland" "OECD"      2020 2018 565 561.64
"Europe" "Finland" "OECD"      2020 2019 646 161.62
"Europe" "Finland" "OECD"      2020 2020 194 133.69
"Europe" "Finland" "OECD"      2021 2018 529  39.76
"Europe" "Finland" "OECD"      2021 2019 800 680.12
"Europe" "Finland" "OECD"      2021 2020 418 399.19
"Europe" "Finland" "OECD"      2022 2018 591 253.12
"Europe" "Finland" "OECD"      2022 2019 457 272.58
"Europe" "Finland" "OECD"      2022 2020 157  105.1
"Europe" "Finland" "IMF"       2020 2018 860 445.03
"Europe" "Finland" "IMF"       2020 2019 108  47.72
"Europe" "Finland" "IMF"       2020 2020 523 500.58
"Europe" "Finland" "IMF"       2021 2018 560  81.47
"Europe" "Finland" "IMF"       2021 2019 830 664.64
"Europe" "Finland" "IMF"       2021 2020 903 762.62
"Europe" "Finland" "IMF"       2022 2018 179 167.73
"Europe" "Finland" "IMF"       2022 2019 137  98.98
"Europe" "Finland" "IMF"       2022 2020 666 524.86
"Europe" "Finland" "WorldBank" 2020 2018 319 146.01
"Europe" "Finland" "WorldBank" 2020 2019 401 219.56
"Europe" "Finland" "WorldBank" 2020 2020 711  45.35
"Europe" "Finland" "WorldBank" 2021 2018 828  20.97
"Europe" "Finland" "WorldBank" 2021 2019 180   66.3
"Europe" "Finland" "WorldBank" 2021 2020 682  92.57
"Europe" "Finland" "WorldBank" 2022 2018 254   81.2
"Europe" "Finland" "WorldBank" 2022 2019 619 159.08
"Europe" "Finland" "WorldBank" 2022 2020 191  184.4
end


/* Run regressions */

regress gdp population i.year_publication, vce(cluster year_publication)
reghdfe gdp population, absorb(year_publication) vce(cluster year_publication)

[Andrew Musau]

If there are no redundant fixed effects, the two commands will output the same standard errors. But since you have the fixed effects nested within cluster, reghdfe adjusts the degrees of freedom to account for this (see highlighted below).

. reghdfe gdp population, absorb(year_publication) vce(cluster year_publication)
(MWFE estimator converged in 1 iterations)

HDFE Linear regression Number of obs = 162
Absorbing 1 HDFE group F( 1, 2) = 139.03
Statistics robust to heteroskedasticity Prob > F = 0.0071
R-squared = 0.3297
Adj R-squared = 0.3170
Within R-sq. = 0.3297
Number of clusters (year_publication) = 3Root MSE = 176.2482

(Std. err. adjusted for 3 clusters in year_publication)
------------------------------------------------------------------------------
| Robust
gdp | Coefficient std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
population | .4861982 .0412343 11.79 0.007 .3087814 .6636149
_cons | 23.00143 22.21942 1.04 0.409 -72.60102 118.6039
------------------------------------------------------------------------------

Absorbed degrees of freedom:
----------------------------------------------------------+
Absorbed FE | Categories - Redundant = Num. Coefs |
------------------+---------------------------------------|
year_publication | 3 3 0 *|
----------------------------------------------------------+
* = FE nested within cluster; treated as redundant for DoF computation

If you want to obtain the same standard errors, specify the option -dof(none)- in reghdfe.

reghdfe gdp population, absorb(year_publication) vce(cluster year_publication) dof(none)

1. Is one implementation “more correct” or can the error be disregarded?

There is a basis for this adjustment, and this is discussed in the reghdfe documentation.

help reghdfe

[Stephan Scheffelberg]

P.S.: Note that standard errors are the same (for population) if I cluster by, say, country:

regress gdp population i.year_publication, vce(cluster country)
reghdfe gdp population, absorb(year_publication) vce(cluster country)

[Stephan Scheffelberg]

Thanks, Andrew Musau, for explaining what is causing this difference, how to “reconcile” it, and referring to further documentation for the details of it.

[Carlo Lazzaro]

Stephan:
as a by-product of the main thread that I find really interesting, it is unusual for a panel dataset to have standard clustered on -timelvar- instead of -panelvar-.
That said, if you cluster your standard errors on -panelid- -xtreg,fe- and -reghdfe- (but not -regress-) give back the same results:

. xtset country_n year_publication
repeated time values within panel
r(451);

. xtset country_n

. xtreg gdp population i.year_publication, fe vce(cluster country_n)

Fixed-effects (within) regression               Number of obs     =        162
Group variable: country_n                       Number of groups  =          6

R-squared:                                      Obs per group:
     Within  = 0.3227                                         min =         27
     Between = 0.6972                                         avg =       27.0
     Overall = 0.3297                                         max =         27

                                                F(3,5)            =      87.71
corr(u_i, Xb) = 0.0149                          Prob > F          =     0.0001

                                  (Std. err. adjusted for 6 clusters in country_n)
----------------------------------------------------------------------------------
                 |               Robust
             gdp | Coefficient  std. err.      t    P>|t|     [95% conf. interval]
-----------------+----------------------------------------------------------------
      population |   .4852409   .0477251    10.17   0.000     .3625595    .6079223
                 |
year_publication |
           2021  |   4.571981    34.1085     0.13   0.899    -83.10671    92.25067
           2022  |   37.15635     27.484     1.35   0.234    -33.49352    107.8062
                 |
           _cons |   9.607808    27.8759     0.34   0.744    -62.04948    81.26509
-----------------+----------------------------------------------------------------
         sigma_u |  17.706981
         sigma_e |  178.33115
             rho |  .00976279   (fraction of variance due to u_i)
----------------------------------------------------------------------------------

. reghdfe gdp population i.year_publication, abs( country_n) vce(cluster country_n)
(MWFE estimator converged in 1 iterations)

HDFE Linear regression                            Number of obs   =        162
Absorbing 1 HDFE group                            F(   3,      5) =      87.71
Statistics robust to heteroskedasticity           Prob > F        =     0.0001
                                                  R-squared       =     0.3355
                                                  Adj R-squared   =     0.3008
                                                  Within R-sq.    =     0.3227
Number of clusters (country_n) =          6       Root MSE        =   178.3312

                                  (Std. err. adjusted for 6 clusters in country_n)
----------------------------------------------------------------------------------
                 |               Robust
             gdp | Coefficient  std. err.      t    P>|t|     [95% conf. interval]
-----------------+----------------------------------------------------------------
      population |   .4852409   .0477251    10.17   0.000     .3625595    .6079223
                 |
year_publication |
           2021  |   4.571981    34.1085     0.13   0.899    -83.10671    92.25067
           2022  |   37.15635     27.484     1.35   0.234    -33.49352    107.8062
                 |
           _cons |   9.607808    27.8759     0.34   0.744    -62.04948    81.26509
----------------------------------------------------------------------------------

Absorbed degrees of freedom:
-----------------------------------------------------+
 Absorbed FE | Categories  - Redundant  = Num. Coefs |
-------------+---------------------------------------|
   country_n |         6           6           0    *|
-----------------------------------------------------+
* = FE nested within cluster; treated as redundant for DoF computation

.

[Maxence Morlet]

Carlo raises a very interesting about standard error clustering. We generally cluster on -panel var- to allow dependence of observations within units across time, however we generally do not allow (that is not great but just the way it is) dependence of observations across clusters within time periods in classical TWFE estimation.