Time dummies dramatically influence Hansen Test

[Guest]

Hello,

I am estimating a model where I have averaged the data in 5 year periods such as there are 5 observations per country. All variables are potentially endogenous. To account for time-specific effects I have included time dummies. However, when I run the system GMM with time dummies in the iv() part, the Hansen test shows that the instrument set is not good. What can be the reason that time dummies affect Hansen's test so significantly? Moreover, would it be correct not to include them in the iv() part?

1) System GMM without time dumies in the iv() part

xtabond2 diff_gdp log_initial log_Vtraded log_prcreditBI log_trade log_govsize log_school log_infl td? , gmm
> (L.( log_initial log_Vtraded log_prcreditBI log_trade log_govsize log_school log_infl) , lag(. .) collapse)
> two robust
Favoring space over speed. To switch, type or click on mata: mata set matafavor speed, perm.
td3 dropped due to collinearity
Warning: Two-step estimated covariance matrix of moments is singular.
Using a generalized inverse to calculate optimal weighting matrix for two-step estimation.
Difference-in-Sargan/Hansen statistics may be negative.

Dynamic panel-data estimation, two-step system GMM
------------------------------------------------------------------------------
Group variable: Country_ Number of obs = 328
Time variable : period Number of groups = 67
Number of instruments = 29 Obs per group: min = 3
Wald chi2(11) = 94.93 avg = 4.90
Prob > chi2 = 0.000 max = 5
--------------------------------------------------------------------------------
| Corrected
diff_gdp | Coef. Std. Err. z P>|z| [95% Conf. Interval]
---------------+----------------------------------------------------------------
log_initial | -.0058507 .0115582 -0.51 0.613 -.0285044 .0168031
log_Vtraded | .0097747 .0048255 2.03 0.043 .0003169 .0192324
log_prcreditBI | -.004026 .0117632 -0.34 0.732 -.0270813 .0190294
log_trade | -.0131724 .0224614 -0.59 0.558 -.0571959 .0308512
log_govsize | -.0157441 .0082085 -1.92 0.055 -.0318325 .0003443
log_school | .0245715 .0386341 0.64 0.525 -.05115 .100293
log_infl | -.0105925 .0047663 -2.22 0.026 -.0199343 -.0012507
td1 | .0193431 .0139906 1.38 0.167 -.0080779 .0467641
td2 | .0104521 .0041221 2.54 0.011 .002373 .0185312
td4 | .0059646 .0052567 1.13 0.257 -.0043384 .0162676
td5 | -.0030673 .0061593 -0.50 0.618 -.0151394 .0090047
_cons | .1274611 .0767582 1.66 0.097 -.0229823 .2779045
--------------------------------------------------------------------------------
Instruments for first differences equation
GMM-type (missing=0, separate instruments for each period unless collapsed)
L(1/4).(L.log_initial L.log_Vtraded L.log_prcreditBI L.log_trade
L.log_govsize L.log_school L.log_infl) collapsed
Instruments for levels equation
Standard
_cons
GMM-type (missing=0, separate instruments for each period unless collapsed)
D.(L.log_initial L.log_Vtraded L.log_prcreditBI L.log_trade L.log_govsize
L.log_school L.log_infl) collapsed
------------------------------------------------------------------------------
Arellano-Bond test for AR(1) in first differences: z = -2.22 Pr > z = 0.026
Arellano-Bond test for AR(2) in first differences: z = -1.52 Pr > z = 0.129
------------------------------------------------------------------------------
Sargan test of overid. restrictions: chi2(17) = 26.25 Prob > chi2 = 0.070
(Not robust, but not weakened by many instruments.)
Hansen test of overid. restrictions: chi2(17) = 23.78 Prob > chi2 = 0.126
(Robust, but weakened by many instruments.)

2) System GMM with time dummies in the iv() part

xtabond2 diff_gdp log_initial log_Vtraded log_prcreditBI log_trade log_govsize log_school log_infl td* , gmm
> (L.( log_initial log_Vtraded log_prcreditBI log_trade log_govsize log_school log_infl) , lag(. .) collapse) i
> v(td*) two robust
Favoring space over speed. To switch, type or click on mata: mata set matafavor speed, perm.
td3 dropped due to collinearity
Warning: Two-step estimated covariance matrix of moments is singular.
Using a generalized inverse to calculate optimal weighting matrix for two-step estimation.
Difference-in-Sargan/Hansen statistics may be negative.

Dynamic panel-data estimation, two-step system GMM

Group variable: Country_ Number of obs = 328
Time variable : period Number of groups = 67
Number of instruments = 33 Obs per group: min = 3
Wald chi2(11) = 81.10 avg = 4.90
Prob > chi2 = 0.000 max = 5

Corrected
diff_gdp Coef. Std. Err. z P>z [95% Conf. Interval]

log_initial -.0087372 .0118574 -0.74 0.461 -.0319773 .0145029
log_Vtraded .010024 .0040155 2.50 0.013 .0021537 .0178943
log_prcreditBI -.010879 .0133469 -0.82 0.415 -.0370385 .0152805
log_trade .0056315 .0234532 0.24 0.810 -.040336 .0515989
log_govsize -.0149861 .0100768 -1.49 0.137 -.0347363 .0047641
log_school .0364438 .0401597 0.91 0.364 -.0422677 .1151553
log_infl -.0107413 .0054915 -1.96 0.050 -.0215044 .0000219
td1 .0227613 .0112086 2.03 0.042 .0007928 .0447298
td2 .0123142 .0050822 2.42 0.015 .0023533 .0222752
td4 .002834 .0058216 0.49 0.626 -.0085762 .0142442
td5 -.005791 .0076746 -0.75 0.451 -.0208329 .0092509
_cons .0740848 .0818758 0.90 0.366 -.0863888 .2345584

Instruments for first differences equation
Standard
D.(td1 td2 td3 td4 td5)
GMM-type (missing=0, separate instruments for each period unless collapsed)
L(1/4).(L.log_initial L.log_Vtraded L.log_prcreditBI L.log_trade
L.log_govsize L.log_school L.log_infl) collapsed
Instruments for levels equation
Standard
td1 td2 td3 td4 td5
_cons
GMM-type (missing=0, separate instruments for each period unless collapsed)
D.(L.log_initial L.log_Vtraded L.log_prcreditBI L.log_trade L.log_govsize
L.log_school L.log_infl) collapsed

Arellano-Bond test for AR(1) in first differences: z = -2.00 Pr > z = 0.046
Arellano-Bond test for AR(2) in first differences: z = -1.50 Pr > z = 0.134

Sargan test of overid. restrictions: chi2(21) = 44.65 Prob > chi2 = 0.002
(Not robust, but not weakened by many instruments.)
Hansen test of overid. restrictions: chi2(21) = 36.83 Prob > chi2 = 0.018
(Robust, but weakened by many instruments.)

Difference-in-Hansen tests of exogeneity of instrument subsets:
GMM instruments for levels
Hansen test excluding group: chi2(14) = 26.41 Prob > chi2 = 0.023
Difference (null H = exogenous): chi2(7) = 10.42 Prob > chi2 = 0.166
iv(td1 td2 td3 td4 td5)
Hansen test excluding group: chi2(17) = 26.68 Prob > chi2 = 0.063
Difference (null H = exogenous): chi2(4) = 10.14 Prob > chi2 = 0.038

Thank you !

[Sebastian Kripfganz]

You should always include the time dummies in the set of instruments. I would suggest to include them as instruments for the level model only. Including time dummies as regressors but not as instruments does not make much sense. In the best case, you just lose some efficiency (which could already lead to substantial differences in small samples). In the worst case, the coefficients of the time dummies might become (asymptotically) unidentified.

I have a section on time effects in my 2019 London Stata Conference presentation, although it does not explicitly address your question:

• Kripfganz, S. (2019). Generalized method of moments estimation of linear dynamic panel data models. Proceedings of the 2019 London Stata Conference.

[Guest]

Thank you Sebastian! But what can be the reason that upon the inclusion of time dummies Hansen test rejects the null hypothesis? (maybe it has to do with the dropped time dummy td3?) Moreover, is the AR(1) test acceptable? I have seen that it usually needs to beAR(1)=0.000

[Guest]

Lastly, I would also like to ask if the above regressions are correctly specifying the instruments used. I am concerned with the fact that usually, one instruments the lagged dependent variable (log_initial) starting with the first lag and other variables starting with the second. I am not sure if this is my case in the above regressions. I have tried to use this regression (below). Maybe it is more appropriate in this context?

xtabond2 diff_gdp log_initial log_Mcap log_Liab log_trade log_school log_govsize log_infl td*, gmm( log_initi
> al , collapse sp) gmm( L.( log_Mcap log_Liab log_trade log_govsize log_school log_infl ), collapse) iv( td*
> ) robust two small
Favoring space over speed. To switch, type or click on mata: mata set matafavor speed, perm.
td4 dropped due to collinearity
Warning: Two-step estimated covariance matrix of moments is singular.
Using a generalized inverse to calculate optimal weighting matrix for two-step estimation.
Difference-in-Sargan/Hansen statistics may be negative.

Dynamic panel-data estimation, two-step system GMM
------------------------------------------------------------------------------
Group variable: Country_ Number of obs = 328
Time variable : period Number of groups = 67
Number of instruments = 34 Obs per group: min = 3
F(11, 66) = 6.29 avg = 4.90
Prob > F = 0.000 max = 5
------------------------------------------------------------------------------
| Corrected
diff_gdp | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
log_initial | -.0254792 .009753 -2.61 0.011 -.0449517 -.0060067
log_Mcap | .0213477 .0106333 2.01 0.049 .0001177 .0425778
log_Liab | -.0036265 .0122586 -0.30 0.768 -.0281016 .0208486
log_trade | -.0120838 .0205278 -0.59 0.558 -.0530688 .0289012
log_school | .0646161 .0426522 1.51 0.135 -.0205418 .1497741
log_govsize | -.0240701 .0200021 -1.20 0.233 -.0640056 .0158654
log_infl | -.0061236 .0090208 -0.68 0.500 -.0241342 .0118871
td1 | .0244532 .0140363 1.74 0.086 -.0035712 .0524777
td2 | .0109251 .006618 1.65 0.104 -.0022881 .0241384
td3 | -.0003078 .0054295 -0.06 0.955 -.0111481 .0105325
td5 | -.008469 .0040234 -2.10 0.039 -.016502 -.0004361
_cons | .1799717 .0885617 2.03 0.046 .0031526 .3567908
------------------------------------------------------------------------------
Instruments for first differences equation
Standard
D.(td1 td2 td3 td4 td5)
GMM-type (missing=0, separate instruments for each period unless collapsed)
L(1/4).(L.log_Mcap L.log_Liab L.log_trade L.log_govsize L.log_school
L.log_infl) collapsed
L(1/4).log_initial collapsed
Instruments for levels equation
Standard
td1 td2 td3 td4 td5
_cons
GMM-type (missing=0, separate instruments for each period unless collapsed)
D.(L.log_Mcap L.log_Liab L.log_trade L.log_govsize L.log_school
L.log_infl) collapsed
D.log_initial collapsed
------------------------------------------------------------------------------
Arellano-Bond test for AR(1) in first differences: z = -2.17 Pr > z = 0.030
Arellano-Bond test for AR(2) in first differences: z = -1.10 Pr > z = 0.271
------------------------------------------------------------------------------
Sargan test of overid. restrictions: chi2(22) = 52.29 Prob > chi2 = 0.000
(Not robust, but not weakened by many instruments.)
Hansen test of overid. restrictions: chi2(22) = 36.95 Prob > chi2 = 0.024
(Robust, but weakened by many instruments.)

Difference-in-Hansen tests of exogeneity of instrument subsets:
GMM instruments for levels
Hansen test excluding group: chi2(15) = 18.07 Prob > chi2 = 0.259
Difference (null H = exogenous): chi2(7) = 18.88 Prob > chi2 = 0.009
gmm(log_initial, collapse eq(diff) lag(1 4))
Hansen test excluding group: chi2(18) = 30.06 Prob > chi2 = 0.037
Difference (null H = exogenous): chi2(4) = 6.88 Prob > chi2 = 0.142
gmm(log_initial, collapse eq(diff) lag(1 4)) eq(level) lag(0 0))
Hansen test excluding group: chi2(21) = 36.84 Prob > chi2 = 0.018
Difference (null H = exogenous): chi2(1) = 0.11 Prob > chi2 = 0.739
iv(td1 td2 td3 td4 td5)
Hansen test excluding group: chi2(18) = 26.32 Prob > chi2 = 0.093
Difference (null H = exogenous): chi2(4) = 10.63 Prob > chi2 = 0.031

[Sebastian Kripfganz]

Your Hansen test result without the time dummy instruments is simply not reliable. Comparing it to the other result does not tell us anything. The AR tests do not seem to be much of a problem here.

A rejection of the Hansen test can have many reasons. Often the model is misspecified, i.e. imporant variables are missing. This could also be interaction terms or lags of the existing variables. See the last section on "model selection" in my 2019 London Stata Conference presentation for a potential approach to deal with that.

The output below the regression results tells you which instruments are used.

[Guest]

Thank you for your advice. I was thinking that another approach may be to cross-sectionally demean the data. Therefore, there would be no need to include time dummies. I used the following commands and obtained the following results. This time Hansen does not reject the null. Would this be a correct way to go?

egen mean_log_Mcap = mean( log_Mcap ), by(Country_) (for all variables)

generate demeaned_log_Mcap = log_Mcap -mean_log_Mcap (for all variables )

xtabond2 demeaned_diff_gdp demeaned_log_initial demeaned_log_Mcap demeaned_log_Liab demeaned_log_trade demean
> ed_log_govsize demeaned_log_school demeaned_log_infl , gmm( demeaned_log_initial , collapse sp) gmm( L.( dem
> eaned_log_Mcap demeaned_log_Liab demeaned_log_trade demeaned_log_govsize demeaned_log_school demeaned_log_inf
> l ), collapse) robust two small
Favoring space over speed. To switch, type or click on mata: mata set matafavor speed, perm.
Warning: Two-step estimated covariance matrix of moments is singular.
Using a generalized inverse to calculate optimal weighting matrix for two-step estimation.
Difference-in-Sargan/Hansen statistics may be negative.

Dynamic panel-data estimation, two-step system GMM
------------------------------------------------------------------------------
Group variable: Country_ Number of obs = 160
Time variable : period Number of groups = 33
Number of instruments = 30 Obs per group: min = 3
F(7, 32) = 5.37 avg = 4.85
Prob > F = 0.000 max = 5
--------------------------------------------------------------------------------------
| Corrected
demeaned_diff_gdp | Coef. Std. Err. t P>|t| [95% Conf. Interval]
---------------------+----------------------------------------------------------------
demeaned_log_initial | -.0699235 .0176928 -3.95 0.000 -.1059626 -.0338844
demeaned_log_Mcap | .018024 .0063162 2.85 0.008 .0051583 .0308898
demeaned_log_Liab | -.0088383 .0105204 -0.84 0.407 -.0302677 .0125911
demeaned_log_trade | .0161355 .0181522 0.89 0.381 -.0208393 .0531104
demeaned_log_govsize | -.0037842 .0150632 -0.25 0.803 -.0344669 .0268984
demeaned_log_school | .0194749 .0297401 0.65 0.517 -.0411037 .0800534
demeaned_log_infl | -.0009668 .0032183 -0.30 0.766 -.0075222 .0055886
_cons | -.0001675 .0001246 -1.34 0.188 -.0004214 .0000863
--------------------------------------------------------------------------------------
Instruments for first differences equation
GMM-type (missing=0, separate instruments for each period unless collapsed)
L(1/4).(L.demeaned_log_Mcap L.demeaned_log_Liab L.demeaned_log_trade
L.demeaned_log_govsize L.demeaned_log_school L.demeaned_log_infl)
collapsed
L(1/4).demeaned_log_initial collapsed
Instruments for levels equation
Standard
_cons
GMM-type (missing=0, separate instruments for each period unless collapsed)
D.(L.demeaned_log_Mcap L.demeaned_log_Liab L.demeaned_log_trade
L.demeaned_log_govsize L.demeaned_log_school L.demeaned_log_infl)
collapsed
D.demeaned_log_initial collapsed
------------------------------------------------------------------------------
Arellano-Bond test for AR(1) in first differences: z = -2.61 Pr > z = 0.009
Arellano-Bond test for AR(2) in first differences: z = 1.03 Pr > z = 0.302
------------------------------------------------------------------------------
Sargan test of overid. restrictions: chi2(22) = 38.38 Prob > chi2 = 0.017
(Not robust, but not weakened by many instruments.)
Hansen test of overid. restrictions: chi2(22) = 28.16 Prob > chi2 = 0.171
(Robust, but weakened by many instruments.)

Difference-in-Hansen tests of exogeneity of instrument subsets:
GMM instruments for levels
Hansen test excluding group: chi2(15) = 20.22 Prob > chi2 = 0.164
Difference (null H = exogenous): chi2(7) = 7.94 Prob > chi2 = 0.338
gmm(demeaned_log_initial, collapse eq(diff) lag(1 4))
Hansen test excluding group: chi2(18) = 23.04 Prob > chi2 = 0.189
Difference (null H = exogenous): chi2(4) = 5.11 Prob > chi2 = 0.276
gmm(demeaned_log_initial, collapse eq(diff) lag(1 4)) eq(level) lag(0 0))
Hansen test excluding group: chi2(21) = 26.90 Prob > chi2 = 0.174
Difference (null H = exogenous): chi2(1) = 1.26 Prob > chi2 = 0.262

[Sebastian Kripfganz]

You have demeaned the variables by country. That is effectively a fixed-effects transformation (with country-fixed effects, not time-fixed effects). With variables that are not strictly exogenous, this invalidates the instrumental variables. The Hansen test may not detect this invalidity because it is only a test of the overidentifying restrictions, based on the assumption that at least as many valid instruments are available as there are regressors. The latter requirement is likely violated in your case.

If anything, you should demean by period.

[Guest]

Thank you! Another question related to the same model. I have tried to group the countries into more homogenous groups to obtain more economically significant results. Although my Hansen cannot reject the null hypothesis, AR(1) becomes larger = 0.062 and Ar(2)= 979. Does this indicate a problem or one can accept such results?

xtabond2 diff_gdp log_initial log_Mcap log_Liab log_trade log_school log_govsize log_infl td*, gmm( log_initi
> al , collapse sp) gmm( L.( log_Mcap log_Liab log_trade log_govsize log_school log_infl ), collapse) iv( td*
> ) robust two small
Favoring space over speed. To switch, type or click on mata: mata set matafavor speed, perm.
td2 dropped due to collinearity
Warning: Number of instruments may be large relative to number of observations.
Warning: Two-step estimated covariance matrix of moments is singular.
Using a generalized inverse to calculate optimal weighting matrix for two-step estimation.
Difference-in-Sargan/Hansen statistics may be negative.

Dynamic panel-data estimation, two-step system GMM
------------------------------------------------------------------------------
Group variable: Country_ Number of obs = 160
Time variable : period Number of groups = 33
Number of instruments = 34 Obs per group: min = 3
F(11, 32) = 34.48 avg = 4.85
Prob > F = 0.000 max = 5
------------------------------------------------------------------------------
| Corrected
diff_gdp | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
log_initial | -.0108161 .0085561 -1.26 0.215 -.0282444 .0066121
log_Mcap | .0029089 .0072577 0.40 0.691 -.0118746 .0176924
log_Liab | -.0101865 .0105349 -0.97 0.341 -.0316455 .0112725
log_trade | .0107354 .0097941 1.10 0.281 -.0092146 .0306853
log_school | .0400703 .0233032 1.72 0.095 -.0073967 .0875373
log_govsize | .0007092 .0124605 0.06 0.955 -.0246721 .0260904
log_infl | -.0005491 .0050937 -0.11 0.915 -.0109245 .0098264
td1 | -.0057598 .0053446 -1.08 0.289 -.0166465 .0051269
td3 | -.0054331 .0042171 -1.29 0.207 -.014023 .0031569
td4 | -.006852 .0052233 -1.31 0.199 -.0174916 .0037875
td5 | -.0181138 .0077344 -2.34 0.026 -.0338683 -.0023593
_cons | .025506 .1402732 0.18 0.857 -.2602211 .3112332
------------------------------------------------------------------------------
Instruments for first differences equation
Standard
D.(td1 td2 td3 td4 td5)
GMM-type (missing=0, separate instruments for each period unless collapsed)
L(1/4).(L.log_Mcap L.log_Liab L.log_trade L.log_govsize L.log_school
L.log_infl) collapsed
L(1/4).log_initial collapsed
Instruments for levels equation
Standard
td1 td2 td3 td4 td5
_cons
GMM-type (missing=0, separate instruments for each period unless collapsed)
D.(L.log_Mcap L.log_Liab L.log_trade L.log_govsize L.log_school
L.log_infl) collapsed
D.log_initial collapsed
------------------------------------------------------------------------------
Arellano-Bond test for AR(1) in first differences: z = -1.87 Pr > z = 0.062
Arellano-Bond test for AR(2) in first differences: z = 0.03 Pr > z = 0.979
------------------------------------------------------------------------------
Sargan test of overid. restrictions: chi2(22) = 43.85 Prob > chi2 = 0.004
(Not robust, but not weakened by many instruments.)
Hansen test of overid. restrictions: chi2(22) = 25.91 Prob > chi2 = 0.255
(Robust, but weakened by many instruments.)

Difference-in-Hansen tests of exogeneity of instrument subsets:
GMM instruments for levels
Hansen test excluding group: chi2(15) = 15.40 Prob > chi2 = 0.423
Difference (null H = exogenous): chi2(7) = 10.52 Prob > chi2 = 0.161
gmm(log_initial, collapse eq(diff) lag(1 4))
Hansen test excluding group: chi2(18) = 23.43 Prob > chi2 = 0.175
Difference (null H = exogenous): chi2(4) = 2.49 Prob > chi2 = 0.647
gmm(log_initial, collapse eq(diff) lag(1 4)) eq(level) lag(0 0))
Hansen test excluding group: chi2(21) = 25.91 Prob > chi2 = 0.210
Difference (null H = exogenous): chi2(1) = -0.00 Prob > chi2 = 1.000
iv(td1 td2 td3 td4 td5)
Hansen test excluding group: chi2(18) = 24.99 Prob > chi2 = 0.125
Difference (null H = exogenous): chi2(4) = 0.92 Prob > chi2 = 0.921

[Sebastian Kripfganz]

I cannot give you a definitive answer. There is no threshold that sharply separates good models from bad models. Personally, I would not worry much about an AR(1) p-value of 0.06.