robustness test in reghdfe

Khati Zolfaghari

Join Date: Mar 2021
Posts: 221

robustness test in reghdfe

01 Aug 2021, 04:06

hello everyone, I am using panel data fixed effect , and in term of the checking the robustness test I estimate the my model before and after using the VCE , in terms of the robustness test in reghdfe, which kind of examination can we used?My supervisor asked my if I tested the robustness of my models.
1. What does he exactly mean?
2. How can I test this? to see all aspect of the robustness test( heteroskedastisity and misespesificatio or other aspect)

Code:

local hmax = 1

forv h = 0/`hmax' {

reghdfe lnfertility`h' l(0/2)dum_recession l(1/2)lnfertility , absorb( i.ifscode i.year)

}

Code:

HDFE Linear regression                            Number of obs   =      2,974
Absorbing 2 HDFE groups                           F(   5,   2807) =    3531.65
                                                  Prob > F        =     0.0000
                                                  R-squared       =     0.9996
                                                  Adj R-squared   =     0.9996
                                                  Within R-sq.    =     0.8628
                                                  Root MSE        =     0.0300

-------------------------------------------------------------------------------
 lnfertility1 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
dum_recession |
          --. |  -.0041092   .0018641    -2.20   0.028    -.0077644   -.0004539
          L1. |  -.0017355   .0018558    -0.94   0.350    -.0053743    .0019033
          L2. |  -.0023126   .0018611    -1.24   0.214    -.0059618    .0013366
              |
  lnfertility |
          L1. |   1.480848   .0269853    54.88   0.000     1.427935    1.533761
          L2. |  -.5983321   .0264009   -22.66   0.000    -.6500993    -.546565
              |
        _cons |   1.976596   .1139087    17.35   0.000     1.753243    2.199949
-------------------------------------------------------------------------------

Absorbed degrees of freedom:
-----------------------------------------------------+
 Absorbed FE | Categories  - Redundant  = Num. Coefs |
-------------+---------------------------------------|
     ifscode |       142           0         142     |
        year |        21           1          20     |
-----------------------------------------------------+

and with using VCE

Code:

local hmax = 1


forv h = 0/`hmax' {

reghdfe lnfertility`h' l(0/2)dum_recession l(1/2)lnfertility , absorb( i.ifscode i.year) vce(cluster ifscode)

}

Code:

HDFE Linear regression                            Number of obs   =      2,974
Absorbing 2 HDFE groups                           F(   5,    141) =    1131.20
Statistics robust to heteroskedasticity           Prob > F        =     0.0000
                                                  R-squared       =     0.9996
                                                  Adj R-squared   =     0.9996
                                                  Within R-sq.    =     0.8628
Number of clusters (ifscode) =        142         Root MSE        =     0.0300

                               (Std. Err. adjusted for 142 clusters in ifscode)
-------------------------------------------------------------------------------
              |               Robust
 lnfertility1 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
dum_recession |
          --. |  -.0041092   .0023937    -1.72   0.088    -.0088414     .000623
          L1. |  -.0017355   .0021453    -0.81   0.420    -.0059766    .0025056
          L2. |  -.0023126   .0020043    -1.15   0.251     -.006275    .0016499
              |
  lnfertility |
          L1. |   1.480848   .1020886    14.51   0.000     1.279026     1.68267
          L2. |  -.5983321    .095414    -6.27   0.000    -.7869592   -.4097051
              |
        _cons |   1.976596    .226157     8.74   0.000     1.529499    2.423693
-------------------------------------------------------------------------------

Absorbed degrees of freedom:
-----------------------------------------------------+
 Absorbed FE | Categories  - Redundant  = Num. Coefs |
-------------+---------------------------------------|
     ifscode |       142         142           0    *|
        year |        21           0          21     |
-----------------------------------------------------+
* = FE nested within cluster; treated as redundant for DoF computation

if I understood correctly, so it seems that without using the standard error the result is more reliable.

thank you so much

Last edited by Khati Zolfaghari; 01 Aug 2021, 04:15.

Tags: None

Carlo Lazzaro

Join Date: Apr 2014
Posts: 17851

01 Aug 2021, 04:40

Khati:
in all likelihood (but please double-check with her/him) your supervisor wants to know whether a comprehensive post-estimation routine was performed.
As you surmised, it includes checking for potential nuisances, such as heteroskedasticity, autocorrellation and, much more substantive, model misspecification.
As your T dimension is relevant (21 years) it seems very unlikely that you do not have heteroshedastcity and/or aotocorrelation issues; hence, I would stick with the clustered standard error.
As far as the most relevant source of problems (ie, model misspecification) is concerned, you can check whether including fitted and sq_fitted values in an augmented regression gives some clues of statistical significance, as it is the case in the following toy-example:

Code:

. use "https://www.stata-press.com/data/r16/nlswork.dta"
. reghdfe ln_wage c.age##c.age fitted sq_fitted , absorb( i.idcode i.year) vce(cluster idcode)
(dropped 551 singleton observations)
(converged in 9 iterations)
note: fitted omitted because of collinearity

HDFE Linear regression                            Number of obs   =     27,959
Absorbing 2 HDFE groups                           F(   3,   4158) =      37.34
Statistics robust to heteroskedasticity           Prob > F        =     0.0000
                                                  R-squared       =     0.6598
                                                  Adj R-squared   =     0.6001
                                                  Within R-sq.    =     0.0129
Number of clusters (idcode)  =      4,159         Root MSE        =     0.3011

                             (Std. Err. adjusted for 4,159 clusters in idcode)
------------------------------------------------------------------------------
             |               Robust
     ln_wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .3610546   .0685459     5.27   0.000     .2266681    .4954412
             |
 c.age#c.age |  -.0050048   .0009355    -5.35   0.000    -.0068389   -.0031708
             |
      fitted |          0  (omitted)
   sq_fitted |  -1.730591   .4047613    -4.28   0.000     -2.52414   -.9370428
------------------------------------------------------------------------------

Absorbed degrees of freedom:
---------------------------------------------------------------+
 Absorbed FE |  Num. Coefs.  =   Categories  -   Redundant     |
-------------+-------------------------------------------------|
      idcode |            0            4159           4159 *   |
        year |           14              15              1     |
---------------------------------------------------------------+
* = fixed effect nested within cluster; treated as redundant for DoF computation

. predict fitted, xb
(24 missing values generated)

. gen sq_fitted=fitted^2
(24 missing values generated)

. reghdfe ln_wage c.age##c.age fitted sq_fitted , absorb( i.idcode i.year) vce(cluster idcode)
(dropped 551 singleton observations)
(converged in 9 iterations)
note: fitted omitted because of collinearity

HDFE Linear regression                            Number of obs   =     27,959
Absorbing 2 HDFE groups                           F(   3,   4158) =      37.34
Statistics robust to heteroskedasticity           Prob > F        =     0.0000
                                                  R-squared       =     0.6598
                                                  Adj R-squared   =     0.6001
                                                  Within R-sq.    =     0.0129
Number of clusters (idcode)  =      4,159         Root MSE        =     0.3011

                             (Std. Err. adjusted for 4,159 clusters in idcode)
------------------------------------------------------------------------------
             |               Robust
     ln_wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .3610546   .0685459     5.27   0.000     .2266681    .4954412
             |
 c.age#c.age |  -.0050048   .0009355    -5.35   0.000    -.0068389   -.0031708
             |
      fitted |          0  (omitted)
   sq_fitted |  -1.730591   .4047613    -4.28   0.000     -2.52414   -.9370428
------------------------------------------------------------------------------

Absorbed degrees of freedom:
---------------------------------------------------------------+
 Absorbed FE |  Num. Coefs.  =   Categories  -   Redundant     |
-------------+-------------------------------------------------|
      idcode |            0            4159           4159 *   |
        year |           14              15              1     |
---------------------------------------------------------------+
* = fixed effect nested within cluster; treated as redundant for DoF computation

. test sq_fitted=0

 ( 1)  sq_fitted = 0

       F(  1,  4158) =   18.28
            Prob > F =    0.0000
*as per -test- outcome, the model is misspecified*
.

Kind regards,
Carlo
(Stata 19.0)

Comment

Khati Zolfaghari

Join Date: Mar 2021
Posts: 221

01 Aug 2021, 06:49

@Carlo Lazzaro thank you so much for your reply, I predict the fitted and sure of it but I do not know what is my problem that I cannot reject the model:

Code:

HDFE Linear regression                            Number of obs   =      2,974
Absorbing 2 HDFE groups                           F(   6,    141) =    1143.22
Statistics robust to heteroskedasticity           Prob > F        =     0.0000
                                                  R-squared       =     0.9996
                                                  Adj R-squared   =     0.9996
                                                  Within R-sq.    =     0.8630
Number of clusters (ifscode) =        142         Root MSE        =     0.0300

                               (Std. Err. adjusted for 142 clusters in ifscode)
-------------------------------------------------------------------------------
              |               Robust
 lnfertility1 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
dum_recession |
          --. |    -.00349   .0024091    -1.45   0.150    -.0082527    .0012726
          L1. |  -.0014424    .002198    -0.66   0.513    -.0057878     .002903
          L2. |  -.0019087   .0020819    -0.92   0.361    -.0060244    .0022069
              |
  lnfertility |
          L1. |   1.290468   .3816996     3.38   0.001     .5358736    2.045062
          L2. |  -.5241403   .1772709    -2.96   0.004    -.8745927   -.1736879
              |
      fittedd |          0  (omitted)
   sq_fittedd |    .004005   .0073136     0.55   0.585    -.0104535    .0184635
        _cons |   2.791445   1.603477     1.74   0.084    -.3785193    5.961408
-------------------------------------------------------------------------------

Absorbed degrees of freedom:
-----------------------------------------------------+
 Absorbed FE | Categories  - Redundant  = Num. Coefs |
-------------+---------------------------------------|
     ifscode |       142         142           0    *|
        year |        21           0          21     |
-----------------------------------------------------+
* = FE nested within cluster; treated as redundant for DoF computation

then:

Code:

 ( 1)  sq_fittedd = 0

       F(  1,   141) =    0.30
            Prob > F =    0.5848

thank you so much for your guidance.
.

Last edited by Khati Zolfaghari; 01 Aug 2021, 06:58.

Comment

Carlo Lazzaro

Join Date: Apr 2014

Posts: 17851
#4

01 Aug 2021, 09:09

Khati:
as per -test- outcome you do not have any issue with your model specification.

Kind regards,
Carlo
(Stata 19.0)
Comment
Khati Zolfaghari

Join Date: Mar 2021

Posts: 221
#5

01 Aug 2021, 09:32

@Carlo Lazzaro
Join Date: Apr 2014 thank you so much for your reply. so my model is correct. but why when I do not use "VCE" the t_test is larger and it seems better? and how can I test the heteroskedasticity in the "reghdfe " ? because "xttset3 " cannot be used?

thank you so much .

Last edited by Khati Zolfaghari; 01 Aug 2021, 09:46.
Comment
Carlo Lazzaro

Join Date: Apr 2014

Posts: 17851
#6

01 Aug 2021, 09:52

Khati:
- clustered standard errors take both autocorrelation and heteroskedasticity into account: so no gain in testing heteroskedasticity here;
- I would find hard to believe that, in your case, default standard errors are the way to go. Please consider that the seemingly "better" ttest are pretty surely misleading.

Kind regards,
Carlo
(Stata 19.0)
Comment
Khati Zolfaghari

Join Date: Mar 2021

Posts: 221
#7

01 Aug 2021, 10:01

@Carlo Lazzaro thank you so much for replying so soon. really thank you.

Best regards,
Comment

Announcement

robustness test in reghdfe

Comment

Comment

Comment

Comment

Comment

Comment