testing to choose between nonnested multivariate fixed effects models

[Neil Meredith]

I am currently estimating two panel models using fixed effects. The models are identical except that one model includes a standard capital ratio as an independent variable while the other model includes a risk-based capital ratio as an independent variable. I am attempting to determine statistically which independent variable is better at explaining variation in my dependent variable. I found in William Greene's fifth edition Econometric Analysis textbook that the J-test and Cox test can be used to evaluate nonnested models to determine which model is more appropriate. Both tests are easily implemented in Stata with cross section or time series data using the nnest user written command. In my case, however, I am using panel data with fixed effects, which does not work with nnest. My question is, does anyone know of a test that can evaluate which of the two models is more appropriate for panel data with fixed effects applied? If so, what is an example command for the test? Or should I just rely on ad-hoc methods (e.g. comparing R-squared values)?

[Joao Santos Silva]

Dear Neil,

If I am not mistaken, there is a very simple way of doing what you want: just estimate a model including both the standard capital ratio and the risk-based capital ratio as regressors and see whether the associated coefficients are significant. As usual, you can get 3 different outcomes:

a) both regressors are significant, which implies that you reject the two original models.
b) neither regressor is significant, which implies that you cannot reject either of the original models.
c) one regressor is significant and the other is not, which implies that you reject one of the original models but not the other.

Would this solve your problem?

Joao

[Neil Meredith]

Joao,

Thank you for your response. According to Greene's (2003) fifth edition Econometric Analysis on p. 154, the problem with including both regressors in the same model is that the coefficients of the other common regressors "remain a mixture of parts of [the coefficients for the risk-based capital ratio and the standard capital ratio]" and a F-test does not establish that the parts are zero. Basically, Greene objects to the approach you are suggesting. That being said, I had already run what you suggested and one regressor, the standard capital ratio, is significant while the other is not. I do not believe that is enough though to satisfy the reviewer (who already saw the combined model results) comment I am seeking to address at this point. Unfortunately, the reviewer's comment has not provided me with any guidance beyond suggesting that I run a formal statistical test to determine if my R-squared values for the two nonnested panel models are different. To my knowledge, unless I am overlooking something like a Chi-square test, t-test, F-test, etc., no test exists for doing what the reviewer suggests.

[Joao Santos Silva]

Dear Neil,

I am afraid I do not have Green's book available so I cannot really comment on what it says.

However, the procedure I suggested is certainly valid; it is an application of the "encompassing approach" of Mizon and Richard (Mizon, G.E. and Richard, J.-F. (1986). "The encompassing principle and its application to non-nested hypothesis tests", Econometrica 54, 657–678.) and it is described for example here (see the discussion around equations 3.23 and 3.24).

Actually, in your case, I believe this approach is also equivalent to the J test proposed by Davidson and MacKinnon (Davidson, R. and MacKinnon, J.G. (1981). "Several tests for model specification in the presence of alternative hypotheses," Econometrica 49, 781-793.).

I am not aware of any tests to check whether R2s are equal and it is not obvious to me that such tests would be interesting, but hopefully, these references will be enough to satisfy the referee.

Best regards,

Joao

[Neil Meredith]

Joao,

Upon closer inspection, I think you are correct. Thank you for getting back to me and for the references. I believe this will satisfy the referee. Thank you again.

Best,
Neil

[Joao Santos Silva]

My pleasure

Joao