Problem with instruments validity after GMM estimation

[Ryan Marapin]

Dear Statalist Forum,

I am currently using Stata version 14.2, and I am using the user-written command "xtabond2" by professor D. Roodman. After reading his paper "How to Do xtabond2: An Introduction to “Difference” and “System” GMM in Stata" I have succesfully managed to estimate my dynamic panel model. I am using a panel dataset of 5500 banks and 13 years. My problem that I have now is that after I use the xtabond2 command, I get proper significant coefficients, and I don't reject the AR(4) test.

However, the Hansen test of overidentifying restrictions has a p-value of 0, meaning I reject the test that my instruments are valid for this model. I have little instruments compared to observations, and I tried to reduce the amount of instruments as well by using collapse, however the Hansen test keeps getting rejected. I am using lagged values of the endogenous variables as instruments, as I am following the methodology of Delis, M., & Kouretas, G. (2011). Interest Rates and Bank Risk-Taking. Journal of Banking and Finance, 35, 840-855.

Any help on how I can overcome this issue of the rejection of the Hansen test of overidentifying restrictions, i.e. my problem of invalid instruments? Does this result render my GMM model useless?

Here is my stata command below:

xtabond2 rwata l.rwata TGAP banksize efficiency offbalancesheet ETA laggedprofitability CPI economicgrowth bankimportance , gmm (rwata banksize efficiency offbalancesheet ETA laggedprofitability, lag (4 4)) iv(TGAP CPI economicgrowth bankimportance ) robust artests (4)

Dependent variable: rwata
Endogenous variables: rwata banksize efficiency offbalancesheet ETA lagged profitability
Exogenous variables: TGAP CPI economic growth bankimportance


Which gives me the following output for the tests:


Dynamic panel-data estimation, one-step system GMM
------------------------------------------------------------------------------
Group variable: bankid Number of obs = 65029
Time variable : year Number of groups = 6587
Number of instruments = 99 Obs per group: min = 3
Wald chi2(10) = 6110.26 avg = 9.87
Prob > chi2 = 0.000 max = 11

Instruments for first differences equation
Standard
D.(TGAP CPI economicgrowth bankimportance)
GMM-type (missing=0, separate instruments for each period unless collapsed)
L4.(rwata banksize efficiency offbalancesheet ETA laggedprofitability)
Instruments for levels equation
Standard
TGAP CPI economicgrowth bankimportance
_cons
GMM-type (missing=0, separate instruments for each period unless collapsed)
DL3.(rwata banksize efficiency offbalancesheet ETA laggedprofitability)
------------------------------------------------------------------------------
Arellano-Bond test for AR(1) in first differences: z = -25.82 Pr > z = 0.000
Arellano-Bond test for AR(2) in first differences: z = 3.18 Pr > z = 0.001
Arellano-Bond test for AR(3) in first differences: z = -2.16 Pr > z = 0.031
Arellano-Bond test for AR(4) in first differences: z = 0.61 Pr > z = 0.545
------------------------------------------------------------------------------
Sargan test of overid. restrictions: chi2(88) =1992.04 Prob > chi2 = 0.000
(Not robust, but not weakened by many instruments.)
Hansen test of overid. restrictions: chi2(88) =1028.01 Prob > chi2 = 0.000
(Robust, but weakened by many instruments.)

Difference-in-Hansen tests of exogeneity of instrument subsets:
GMM instruments for levels
Hansen test excluding group: chi2(41) = 592.39 Prob > chi2 = 0.000
Difference (null H = exogenous): chi2(47) = 435.61 Prob > chi2 = 0.000
iv(TGAP CPI economicgrowth bankimportance)
Hansen test excluding group: chi2(84) = 982.82 Prob > chi2 = 0.000
Difference (null H = exogenous): chi2(4) = 45.19 Prob > chi2 = 0.000


I hope I followed the posting rules set by Stalist as best possible. Any help is highly appreciated.

Thank you for your time.

Kind regards,

Ryan

[Sebastian Kripfganz]

A rejection of the validity of the overidentifying restrictions can have many reasons. Different sorts of model misspecification could be the cause. Some generic ideas:

1. If you are not specifically interested in the coefficients of your "global" variables CPI and economic growth, you may just use time dummies that capture any macroeconomic fluctuations which affect all banks in the same way.
2. The fourth lag of your endogenous variables may not have strong correlation with the endogenous variables themselves and thus might be a weak instrument. It is probably a better idea to try to get rid of the serial correlation such that you can use already the second lag as a valid instrument. Using time dummies could help in that regard. Alternatively, using also a second lag of the dependent variable could help to reduce the residual autocorrelation.
3. I generally advise against using the gmm() and the iv() option without explicit reference to a particular equation. In particular, iv() without suboption equation() is not doing what you believe it does. Please see my comment in the following Statalist discussion and the further links therein: Help with system GMM and xtabond2.
4. Maybe some of your strictly exogenous variables are actually predetermined. This would make your instruments invalid. Yet, I cannot judge this because I do not know the literature in your field. Moreover, relaxing too many assumptions is also not a good idea. The more variables you assume to be endogenous or predetermined, the more likely it is that you have many weak instruments.
5. In particular given your large number of observations, I would recommend to use the asymptotically efficient twostep estimator.

[Ryan Marapin]

Dear professor Kripfganz,

Thank you kindly for the reply, I highly appreciate it. I have started trying out some of your ideas, unfortunately not much has changed, I am still rejecting the Hansen and Sargan test with a p-value of 0.000. I will keep trying however and carefully analyze what might be wrong. Just a question however, does this problem render my GMM model useless, or could I still use this model but note that weak instruments are used in this model?

Kind regards,

Ryan

[Sebastian Kripfganz]

If you take it at face value, a rejection of the null hypothesis underlying the Hansen test implies that some characteristics of your data are not in line with the model assumptions. Unfortunately, it remains silent about the source of model misspecification. Your difference-in-Hansen statistics also do not provide further insides. (Maybe they do when you separate the specification of the instruments by equation?)

Say, there might be an important omitted variable that is correlated with your instruments. Modifying the lags of the instruments and similar changes may not be helpful in addressing such a source of model misspecification.

I would not say, that your model is generally useless but you need to be cautious about the interpretation of the results and you should demonstrate that you have considered different sorts of potential model misspecification. (Others might have different opinions about this.)

A final remark: Keep in mind that statistical tests can give false positives. A correctly sized test (at the 5% level) rejects the null hypothesis in 5% of the cases (assuming you could repeatedly draw sample data from the population) even though it is actually true. Your sample might just be such an extreme draw from the underlying distribution. That is not really a justification for ignoring the result of the test, but if you have a sound theoretical reasoning for the choice of your moment conditions and if you can show that the results are rather robust to different specifications, you might get away with such a rejection of the Hansen test.

[Ryan Marapin]

Dear professor Kripfganz,

Thank you for your very informative reply,

you have helped me very much, I am grateful.

Regards,

Ryan