working with subsets of (weighted) survery data

[patrick thornton]

I have a large (4 million) record weighted survey data set. I am only concerned with a subset of the data. I want to delete records/observations which are not part of my analysis so I can work with a smaller list of observations. I am concerned that if I start deleting observations it may affect the weights of the remaining observations undermining the validity of the remaining weighted sample. Is this a valid concern, or will the weights applied to the remaining observations retain their original validity? I realize I could use a (subpop) approach, but it would be simpler to cull unwanted observations prior to analysis.

[Dick Campbell]

It is a valid concern. You don't say much about the weights, but assuming that they were derived from a complex survey the conventional statistical wisdom is that you have to use the subpop approach if you want valid standard errors. Although there are many cases where you will get the same results, or very closely similar results, between a correct analysis and just dropping the irrelevant cases, the conditions under which that will be true are somewhat complex. The UCLA web site has examples at http://www.ats.ucla.edu/stat/stata/f...ata_subpop.htm. You might also want to read the article by Brady West et al in the Stata Journal, Vol 8 No. 4. Yu can access it free at http://www.stata-journal.com/article...article=st0153.

[patrick thornton]

Thank you, Dr Campbell, I feared as much but will make subpop work. BTW, I think you did a guest lecture on missing data for a PhD seminar I took a couple years ago at UIC College of Nursing. Patrick Thornton

[Dick Campbell]

Your're welcome and, yes, that was me. I've learned a bit since then.