Using vce(bobust) in regression leads to differen significance

[Niklas Adolph]

I have a short question which I did not find an answer with a valid source yet.
I've made a regression which happens to have heteroskedasticity in it. I used to robust command and it turned out that one of my variables is now significant even though it wasn't before.
Is this a violation? As I thought the robust values are better…
I did not find any literature about this - that's why I ask and I would be glad if someone has a source/tip for this kind of problem for me.
Thanks a lot!

[Marcos Almeida]

Please read this text.

That being said, the best approach to entice discussion - and useful replies - in the Forum is sharing command/data/output, as underlined in the FAQ.

Hopefully that helps.

[Nick Cox]

There would be no point in choosing vce(robust) if it never made any difference to your conclusions. Equally most books worth reading seem to advise not making a fetish of significance levels. They give indications, but don't impel decisions unless someone in power over you is forcing you to be rigid about inference.

More neutrally put, getting the same substantive conclusions is reassuring; getting different conclusions should guide thought on which model better suits the data and ideas on the generating process.

[Joro Kolev]

I believe what the original poster means is that he applied the robust option to a regression, and something which was not significant before, turned out to be significant. Which is puzzling because OLS non-robust standard errors are supposed to over-estimate significance (under-state the size of the standard errors).

It is puzzling, but it happens, and it happens often in practice.

The issue has been discussed at this forum before for sure, for the case of using ,robust cluster() option.

If I am not wrong, for the case of clustered standard errors, there was a clear cut theoretical result which goes something in the lines of:

If you are regressing a dependent variable on a regressor that varies at more aggregate level (say you re regressing personal wage on a variable that varies at county/state level, say state level unemployment) you should be clustering at the level of variation of the main regressor (county / state level unemployment). Moulton pointed this out in two influential papers, and some people call it the Moulton's problem in econometrics. In Moulton's analysis the wrong standard errors come about because there is an unobserved county/state level random effect. In this Moulton model there was a theoretical result in the lines that if the unobserved state/county effect was positively/negatively correlated with your main variable of interest (state level unemployment say) your standard errors will be over/under estimated.

I dont think that there is such theoretical result explaining when robust errors will be smaller than non-robust for general heteroskedasticity. And it is hard to imagine how such a result can be derived because there are so many ways in which heteroskedasticity can come about.

[Marcos Almeida]

I'm just coping the excerpt of the text I shared in #2:

So, if the robust (unclustered) estimates are just a little smaller than the OLS estimates, it may be that the OLS assumptions are true and you are seeing a bit of random variation. If the robust (unclustered) estimates are much smaller than the OLS estimates, then either you are seeing a lot of random variation (which is possible, but unlikely) or else there is something odd going on between the residuals and the x’s.

[Carlo Lazzaro]

Niklas:
interested listers could reply even more positively if you share what you typed and what Stata gave you back via CODE delimiters (as per FAQ). Thanks.