Panel Data Analysis

[Mekky Maher]

Hi,
I running a regression on panel data analysis: pooled OLS, Fixed Effects, and Random effects.
One of the variables is significant in polled ols and random effects, but it is insignificant in the Fixed effects model.
Is there any explanation why the variable (which is time variant) is insignificant only with the fixed effects model??

[Clyde Schechter]

First of all, regardless of the particular analysis, it is important to always remember that the difference between statistically significant and not statistically significant is, itself, not statistically significant. Nor even meaningful in any useful sense. So don't focus on that.

Disregarding the red herring of statistical significance, there is the real question of is there a meaningfully large difference between the coefficient in one model vs. another. It is not at all uncommon for the fixed-effects coefficient to differ from what is obtained with OLS or random effects. The reason is that the fixed-effects model is a very different model from the other two and there is no reason to expect the coefficients to be similar, or even to have the same sign. The fixed-effects model estimates the within-panel effects of the variables, whereas the OLS and random effects models measure a weighted average of the within-panel and between-panel effects.

The following code creates a small data set where it is possible to visualize directly what is going on. And you can see that the within-panel and between-panel effects run in opposite directions in this example:

clear
set obs 5
gen panel_id = _n
expand 2

set seed 1234
by panel_id , sort: gen y = 4*panel_id - _n + 3 + rnormal(0, 0.5)
by panel_id: gen x = panel_id + _n

xtset panel_id

xtreg y x, fe
regress y x

//    GRAPH THE DATA TO SHOW WHAT'S HAPPENING
separate y, by(panel_id)

graph twoway connect y? x || lfit y x

So there is no explanation why the results differ, as there is no reason to think they wouldn't.

[Richard Williams]

Fixed effects models control for the effects of time-invariant variables with time-invariant effects. Once those controls are applied, the effects of the variables that are in the model might diminish.

For example, suppose gender is a common cause of X and Y, and is responsible for all of the correlation between them. If you don't control for gender, X will appear to affect Y. But, once you do control for gender, X will have no effect on Y.

Also, if there is very little variation within a group, e.g. X and Y stay pretty much the same across time, a fixed-effects model will have trouble detecting a relationship between them.

There are other things to consider. Is the effect of X really all that different in all the approaches? For example, if the effect of X is "significant" at the .048 level in one approach but is "insignificant" at the .052 level with another, you probably don't want to make too much of that. I've seen people say things like X has an effect in population A but no effect in population B, but once you look at the actual coefficients in the two groups they aren't very different. The fact that the effect is significant in one group but not the other may just reflect the fact that one group is much larger than the other.

I'm not as hostile to P values as some people are. But I do think you have to be carefull of statements like "Significant here but not there" because such findings may reflect differences in statistical power rather than real differences between the coefficients.

EDIT: Crossed with Clyde's excellent comments.

[Mekky Maher]

Thank you so much, Clyde Schechter and Richard Williams.