Clustering in Multiple Linear Regression

[Resa Pfu]

Dear all,

I have created a multiple linear regression model with a checklist score as dependant variable and independent variables device_group, child_age, education, monthsofexperience. The dataset is an observational dataset where healthworkers were observed when using a medical device in children under five and the checklist score describes the proportion of tasks (out of a total task list of 9) the health workers completed correctly. Because the dataset contains multiple observations for some health workers I have clustered on the health worker ID code. This is my code:

regress chechklist_score i.device_group i.child_age i.education monthsofexperience, cluster (hw_code_numeric)

Now I am wondering what the difference between my model and a multilevel model is and how the interpretation would differ between the two? Compared to a model without clustering, the coefficients have not changed, but only the confidence intervals.

Thanks in advance,

best
Theresa

[Carlo Lazzaro]

Resa:
1) Clustering on -hw_code_numeric- makes sense is some healthworkers have multiple observations (I meant here were measured on the same variables >1 times);
2) given what aboce, you have a panel dataset in which, some individuals have one wave of data only, whereas some of them have two waves of data;
3) if you get evidence of a panel-wise effect you should switch to -xtreg- otherwise you can stick with -regress- with clustered standar errors.

[Resa Pfu]

Thank you Carlo for the stepwise explanation. So to confirm that I understood this correctly, a "panel-wise effect" would be if my coefficients in the clustered model were different from the ones in the regression model without clustering?
Compared to the unclustered model the clustered model has slightly larger confidence intervals, which I thought was due to the robust standard errors?

[Carlo Lazzaro]

Resa:
1) not quite. First, I do not know if are planning to go -xtreg, fe- or -xtreg, re- with clustered standard errors.
After the -re- specification, no mattet the standard errors you invoke, you can go -xttest0- and, if the test outcome reaches statistical significance, there's evidence of panel-wise effect in your dataset.
Unfortunately, there's no a similar test for the -fe- specification. However, there's a workaround (a tad computationally demanding, though), provided by the community-contributed module -mundlak-
No matter the standard errors you invoke, if -test- after -xtreg, re- (and I mean -re-) gives you evidence of joint statistical significance of the means of the time-varying predictors, the main -re- assumption (no correlation between the -ui- term and the vector of predictors) is rejected, and you can safely go -xtreg, fe-;
2) you're right about the effect of non-default standard errors on confidence intervals width.

[Resa Pfu]

Okai so now I have done this:

xtset hw_code_numeric
xtreg Checklist_Score i.device_group i.Child_age_new i.education monthsofexperience, re
xttest0

The test shows Prob > chibar2 = 0.0725

Can I now assume there is no significant panel effect in my data, so the independence assumption is met and I can run my initial regress command with clusters?

[Carlo Lazzaro]

Resa:
yes, pooled OLS with clustered standard errors seems to be the way to go.
However, the null of -xtest0- is not the correlation=0 between -ui- and the vector of the regressors, but -Var(u) = 0-.

[Resa Pfu]

Great thanks a lot for your help Carlo!