Change in dependent variable suggests change in estimation method

[Kieran Davies]

I'd greatly appreciate any help with this.
When regressing the following, a hausman test confirms that fixed effects is preferred to random effects.

xtreg Y L.X1 L.X2 L.X3, fe

However when I change the dependent variable, as below, but keep the same iv's, a hausman test indicates that random effects is preferred.

xtreg Z L.X1 L.X2 L.X3, fe

Theoretically, the individual effects should be correlated with the iv's in both cases so I don't understand why the second regression is suggesting the use of random effects. Could there be an explanation for this?

[Maarten Buis]

The Hausmann test does not tell you what to do, it only tests the hypothesis that the coefficients in a random and fixed effects model are equal. If that hypothesis is true, then a random effects will have the same coefficients as a fixed effects model (obviously) and be more efficient (smaller standard errors). You don't have to do a random effects model, you just miss out on some advantages if you don't. To make things more complex, we don't know if that hypothesis is true, even if the test returns a non-significant result. A non-significant result means that the data contains insufficient evidence to reject the hypothesis, that is not the same as saying that the data contains evidence that the hypothesis is true. Think of the difference between "absence of evidence" and "evidence of absence". Finally, the fact that your theory says that the random effect and the explanatory variables should be correlated means nothing: theories can be wrong (and almost always are wrong), that is why we do empirical research.

So those are a lot of words to say that you have a lot more room for choice than you seem to think, which leaves the question what should you do. If your result consist of two fairly related models, then it would make sense to use the same estimator for both. If there is sufficient within person variance in the dependent and independent variables, then fixed effects models tend to be safer than random effects models. If not, the choice becomes a lot more complicated.

[Kieran Davies]

That's a very helpful explanation, thank you Maarten. However I'm still very perplexed about which to use.
For the first model there is sufficient evidence to reject the null of the Hausman test so, as you mention, its safer to use fixed effects as the estimations are consistent. :

                  chi2(4) = (b-B)'[(V_b-V_B)^(-1)](b-B)
                          =       32.51
                Prob>chi2 =      0.0000

But for the second model there is insufficient evidence and so it seems, as you mention, that random effects would be more efficient in this case.

chi2(4) = (b-B)'[(V_b-V_B)^(-1)](b-B)
                          =        3.83
                Prob>chi2 =      0.4297

The only thing that has changed between these two models is the dependent variable. The dependent variable in the former is GDP forecast error whereas the dependent variable in the latter is budget balance forecast error. Independent variables remain the same. Estimating the second model with random effects, as opposed to fixed effects, provides more intriguing and fruitful results related to my research question. Estimating the first model with random effects would give me inconsistent estimates however.
My instinct tells me to use fixed effects for both models and admit to the second model having inefficient estimates but then my results are mostly uninteresting. Do you think using fixed effects for the former but random effects for the latter could be justified in any way? It seems difficult to justify using different methods when the primary difference between the two is correlation between individual effects and iv's, especially when the iv's are the same in both cases.

[Maarten Buis]

I am not certain what kind of answer you are expecting from me. There are various trade-offs to be made when choosing a model, and you seem to have a fair idea about the main trade-offs in your case. So just look at them, and decide what you find most important.

[Kieran Davies]

No problem, thanks for your time. Have a great day.

[Phil Bromiley]

Coming in late, let me just note that there is no reason that the same rhs variables should result in anything similar when you change the dv. Would you expect the same anything if you try to explain apples and airplanes with year and macro economics?

I'd like to also emphasize (as Maarten notes) the Hausman tests equality of between and within parameter estimates. This makes sense mostly if theory suggests they should be the same. If there is no reason to believe they should be the same, then it is not clear that the test makes much sense. Or rather, following Hausman may help get a consistent estimate of the within model, but you need to care primarily about within. I could have something that changes over time within panels mainly and another dv that doesn't change much within panels but changes a lot between panels. You might also consider the xthybrid approach.