A Question about OLS v.s.HLM

[Minda Tan]

Hi! I am analyzing a two-level dataset, including student level and school level. The coefficient of one interaction variable was not significant in the HLM model but significant at .05 level in the OLS model. I found that the standard error of this variable from HLM is similar to the one from OLS. However, its coefficient in the OLS model was about 5 times larger than in the HLM.

Some statistical papers claimed that comparing to OLS, HLM can give the more accurate standard errors. It seems that in this condition HLM does not have obvious advantages over OLS. So, I'm a little confused that which method I should employ in my study.

Would you please give me some advice?

[Marcos Almeida]

What seems to be the ‘point’ is the (significant) p value.

But it shouldn’t be so. If the real structure of the data is a hierarchical frame, performing OLS regression may be taken as a naive approach, to say the least. You may probably get a higher p-value under robust SEs.

Additionally, let’s keep in mind the within-group correlation, ’dismissed’ under the naive approach. Last but not least, the nested structure of the data, being lost, is a pity.

Unfortunately, there is no command neither output in thr starting message, hence we cannot help but hazard a guess.

[Clyde Schechter]

Adding to Marcos' advice, the discrepancy in the coefficient between the OLS and HLM models in this case means that much of the outcome variation that is actually attributable to unobserved attributes of the school and person were "proxied" by the interaction variable in the OLS model (which failed to account for school and person distinctions).

Given the hierarchical structure of the data, it is not appropriate to use OLS unless the HLM model estimates the school and person level variance components to be very close to zero, negligibly small. That, I believe, will not be the case with your data.

[Minda Tan]

Thank you so much!