Hansen-J-Test Interpretation

[Roman Neuenschwander]

Dear Statalists

I am actually doing a IV regression with one single endogenous regressior (penetration_row) and two instrumental variables (real_exchange_final & tariff_rate_row). I am using the command ivreghdfe and get the following output:

ivreghdfe share_zombiesBH2 (L3.penetration_row = L4.tariff_rate_row L4.real_exchange_final) L3.ln_at L3.age L3.F_E L3.tnic3hhi L3.dtfp4 L3.tangibility, first absorb(year sic) cluster(gvkey)

First-stage regression of L3.penetration_row:

Statistics robust to heteroskedasticity and clustering on gvkey
Number of obs =                   9497
Number of clusters (gvkey) =      1285
-------------------------------------------------------------------------------------
                    |               Robust
 L3.penetration_row |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
--------------------+----------------------------------------------------------------
    tariff_rate_row |
                L4. |  -.5737043   .3361944    -1.71   0.088    -1.232718     .085309
                    |
real_exchange_final |
                L4. |   .0243828    .004553     5.36   0.000     .0154579    .0333077
                    |
              ln_at |
                L3. |  -.0003283    .000159    -2.06   0.039      -.00064   -.0000166
                    |
                age |
                L3. |   .0000254   .0000657     0.39   0.699    -.0001034    .0001542
                    |
                F_E |
                L3. |   .0006308   .0000899     7.02   0.000     .0004547     .000807
                    |
           tnic3hhi |
                L3. |   -.001099   .0009947    -1.10   0.269    -.0030489    .0008508
                    |
              dtfp4 |
                L3. |   .0265433   .0017163    15.47   0.000     .0231791    .0299075
                    |
        tangibility |
                L3. |  -.0040824      .0022    -1.86   0.064    -.0083949    .0002301
-------------------------------------------------------------------------------------
F test of excluded instruments:
  F(  2,  1284) =    14.34
  Prob > F      =   0.0000
Sanderson-Windmeijer multivariate F test of excluded instruments:
  F(  2,  1284) =    14.34
  Prob > F      =   0.0000



Summary results for first-stage regressions
-------------------------------------------

                                           (Underid)            (Weak id)
Variable     | F(  2,  1284)  P-val | SW Chi-sq(  2) P-val | SW F(  2,  1284)
L3.penetrati |      14.34    0.0000 |       28.84   0.0000 |       14.34

NB: first-stage test statistics cluster-robust

Stock-Yogo weak ID F test critical values for single endogenous regressor:
                                   10% maximal IV size             19.93
                                   15% maximal IV size             11.59
                                   20% maximal IV size              8.75
                                   25% maximal IV size              7.25
Source: Stock-Yogo (2005).  Reproduced by permission.
NB: Critical values are for i.i.d. errors only.

Underidentification test
Ho: matrix of reduced form coefficients has rank=K1-1 (underidentified)
Ha: matrix has rank=K1 (identified)
Kleibergen-Paap rk LM statistic          Chi-sq(2)=161.09   P-val=0.0000

Weak identification test
Ho: equation is weakly identified
Cragg-Donald Wald F statistic                                     421.18
Kleibergen-Paap Wald rk F statistic                                14.34

Stock-Yogo weak ID test critical values for K1=1 and L1=2:
                                   10% maximal IV size             19.93
                                   15% maximal IV size             11.59
                                   20% maximal IV size              8.75
                                   25% maximal IV size              7.25
Source: Stock-Yogo (2005).  Reproduced by permission.
NB: Critical values are for Cragg-Donald F statistic and i.i.d. errors.

Weak-instrument-robust inference
Tests of joint significance of endogenous regressors B1 in main equation
Ho: B1=0 and orthogonality conditions are valid
Anderson-Rubin Wald test           F(2,1284)=      3.70     P-val=0.0251
Anderson-Rubin Wald test           Chi-sq(2)=      7.43     P-val=0.0244
Stock-Wright LM S statistic        Chi-sq(2)=     59.52     P-val=0.0000

NB: Underidentification, weak identification and weak-identification-robust
    test statistics cluster-robust

Number of clusters             N_clust  =       1285
Number of observations               N  =       9497
Number of regressors                 K  =          7
Number of endogenous regressors      K1 =          1
Number of instruments                L  =          8
Number of excluded instruments       L1 =          2

IV (2SLS) estimation
--------------------

Estimates efficient for homoskedasticity only
Statistics robust to heteroskedasticity and clustering on gvkey

Number of clusters (gvkey) =      1285                Number of obs =     9497
                                                      F(  7,  1284) =    20.97
                                                      Prob > F      =   0.0000
Total (centered) SS     =  4.652722306                Centered R2   =  -0.0218
Total (uncentered) SS   =  4.652722306                Uncentered R2 =  -0.0218
Residual SS             =  4.754029143                Root MSE      =   .02243

---------------------------------------------------------------------------------
                |               Robust
share_zombies~2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------+----------------------------------------------------------------
penetration_row |
            L3. |   .2754187   .0562641     4.90   0.000      .165039    .3857983
                |
          ln_at |
            L3. |    .000142   .0001487     0.95   0.340    -.0001498    .0004338
                |
            age |
            L3. |  -.0000765   .0000626    -1.22   0.222    -.0001994    .0000463
                |
            F_E |
            L3. |  -.0001973   .0001446    -1.36   0.173     -.000481    .0000863
                |
       tnic3hhi |
            L3. |  -.0025562   .0010006    -2.55   0.011    -.0045192   -.0005931
                |
          dtfp4 |
            L3. |   .0244666   .0030023     8.15   0.000     .0185765    .0303566
                |
    tangibility |
            L3. |  -.0002343   .0021431    -0.11   0.913    -.0044387      .00397
---------------------------------------------------------------------------------
Underidentification test (Kleibergen-Paap rk LM statistic):            161.088
                                                   Chi-sq(2) P-val =    0.0000
------------------------------------------------------------------------------
Weak identification test (Cragg-Donald Wald F statistic):              421.176
                         (Kleibergen-Paap rk Wald F statistic):         14.342
Stock-Yogo weak ID test critical values: 10% maximal IV size             19.93
                                         15% maximal IV size             11.59
                                         20% maximal IV size              8.75
                                         25% maximal IV size              7.25
Source: Stock-Yogo (2005).  Reproduced by permission.
NB: Critical values are for Cragg-Donald F statistic and i.i.d. errors.
------------------------------------------------------------------------------
Hansen J statistic (overidentification test of all instruments):        13.598
                                                   Chi-sq(1) P-val =    0.0002
------------------------------------------------------------------------------
Instrumented:         L3.penetration_row
Included instruments: L3.ln_at L3.age L3.F_E L3.tnic3hhi L3.dtfp4
                      L3.tangibility
Excluded instruments: L4.tariff_rate_row L4.real_exchange_final
Partialled-out:       _cons
                      nb: total SS, model F and R2s are after partialling-out;
                          any small-sample adjustments include partialled-out
                          variables in regressor count K
------------------------------------------------------------------------------

Absorbed degrees of freedom:
-----------------------------------------------------+
 Absorbed FE | Categories  - Redundant  = Num. Coefs |
-------------+---------------------------------------|
        year |        19           0          19     |
         sic |        19           1          18     |
-----------------------------------------------------+

For the Hansen-J statistic I get e value of 13.598 and a P-Value of 0.002. Is it therefore right, that my instruments are not valid and the coefficient of the endogenous regressor on the dependent variable may be biased?

Thanks for your help
Roman

[Joao Santos Silva]

Dear Roman Neuenschwander,

Keep in mind that the J-statistic does not allow you to test if the instruments are valid; that is an identifying assumption. What the test is suggesting is that different instruments identify different parameters; for example, they may be identifying different LATEs. Of course, the instruments can also be invalid, but that is always the case!

Best wishes,

Joao

[Roman Neuenschwander]

Dear Joao Santos Silva

Thank you for your helpful comment.

[Ellen Sterk]

Joao Santos Silva Hi Joao. I have a similar question as Roman, but do not quite understand your reply "What the test is suggesting is that different instruments identify different parameters" - what does that mean exactly and how could I handle that problem? Thank you!

[Joao Santos Silva]

Dear Ellen Sterk,

The test effectively checks whether the estimates obtained each possible just-identified model are significantly different. If different instruments identify different local average treatment effects, the null will be rejected even if each instrument is valid for a particular LATE.

Best wishes,

Joao

[Ellen Sterk]

Joao Santos Silva Ok, I get it now - thank you!
So say the test rejects the null because the instruments identify different LATEs (and not because they are 'invalid'), that is still a problem, right? Is it right that that tells me I shouldn't use this combination of instruments?

[Joao Santos Silva]

That is correct, Ellen Sterk.

[Ellen Sterk]

Joao Santos Silva Ok. Thank you very much!

[Shreya Jain]

Hello @Joao Santos Silva , I have a follow up question here. So if Hansen J test accepts the null hypothesis would that mean the instruments are valid and the results obtained by IV2SLS are the true effect of the endogenous variable on the outcome variable?

[Joao Santos Silva]

Dear Shreya Jain

No, if the null is not rejected that only means that you do not reject that all instruments identify the same parameter, which may or may not be the parameter you want to estimate.

Best wishes,

Joao

[Shreya Jain]

Thank you for your response Joao Santos Silva

[Shreya Jain]

Thank you for your response Joao Santos Silva