inconsistent p-values after clustering

[Pablo Gonzaga Lopes]

Hi everybody.

I am running a large panel regression of firms and quarters and firms are related to two countries (India and Bangladesh). Good econometrics suggest that for the coefficient of interest I should cluster standard errors in the country level (so N=2 clusters). The problem though is that the regression outputs of reg become strange. My coefficient of interest has a t-stats of 9.09 (SE clustered in country level) but the corresponding p-value is about 7% (or significance less then 10%, leading to one * only).

When I cluster SE in the firm level then results become consistent since I have more than 5000 firms. I believe when you cluster the degrees of freedom of the t-distribution become the number of clusters, so that's why for N=2 clusters the t-distribution would be very different than the normal approximation (so the thresholds for the t-stats). However even for N=2 this t-stats should be equivalent to a p-value less than 2% (two-sided).

Can anybody give me a helping hand (or thought)?

Thank you very much!

Pablo Gonzaga

[Bryan Sayer]

I believe the bigger impact is from intra-cluster correlation, rather than degrees of freedom. And usually the correlation is positive, which results in an increase of the variance (se). How do your actual standard errors compare?

[Pablo Gonzaga Lopes]

Thanks Bryan. Actually the t-stats for firm-level cluster is about 6.00 but for this case the p-value is less than 0.01. Is it because for firm level cluster the high number of clusters makes the t distribution very close to normal?

[Bryan Sayer]

Yes, I suspect that is what you are observing.