Questions about statistical significance

[Ella Ki]

Irrelevant question. Not sure if deleting posts is possible.

[Clyde Schechter]

What does this mean and can the two models still be compared to conclude that unobserved heterogeneity is or is not present?

Pooled OLS and fixed effects regression estimate very different models and it is not surprising when their results differ. A simple way to explore unobserved heterogeneity is to look at the last several lines of the -xtreg, fe- output. There you will see estimates for sigma_u, sigma_e, rho, and an F-test that all u_i = 0. Heterogeneity is present unless the following are true: rho is very close to zero, and the F-test that all u_i = 0 is not statistically significant.

Bear in mind that -xtreg, fe- estimates within-panel effects only. It tells you how much differences in X are associated with differences in Y over time within the same panel. But OLS does not distinguish differences within panels from differences between panels. Consequently, not only can the resulting coefficients differ greatly in magnitude, they can actually be opposite in signs. For a simple, crystal clear example, read and run this code:

clear
set obs 5
gen panel_id = _n
expand 2

set seed 1234
by panel_id , sort: gen y = 4*panel_id - _n + 3 + rnormal(0, 0.5)
by panel_id: gen x = panel_id + _n

xtset panel_id 

xtreg y x, fe
regress y x

//    GRAPH THE DATA TO SHOW WHAT'S HAPPENING
separate y, by(panel_id)

graph twoway connect y? x || lfit y x

Also, what does it mean if re-estimating a model with an additional variable renders some existing key variables highly insignificant?

When you change model, results change. It means that some of the variance in your outcome explained by one of the variables in the first model is actually attributable to one of the variables you just added to the model. Those two predictors are correlated with each other and the first was serving as a proxy for the second. When the second variable was added to the model, the first variable only gets "credit" for what it does above and beyond the second.