use of svyset with propensity scores

[frank.lopresti]

I'm working with NAMCS data and using svyset and want to create propensity scores. The treatment will be patients with a particular diagnosis and the outcome
is the length of the visit. What confuses me is the svyset. Can you suggest materials I should read. Thanks, Frank

[Melissa Garrido]

Hi Frank,

When using propensity scores with survey weighted data, it's important to determine whether you prefer to make within-sample inferences (where you can use propensity scores with no adjustment for survey weights) or whether you would like your results to be generalizable to the broader population from which the survey sample was drawn (where you'll need to combine propensity score weights with survey weights).

A good reference on this topic is by DuGoff et al in Health Services Research 2014; 49(1):284-303 (http://www.ncbi.nlm.nih.gov/pubmed/23855598).

Hope this helps,
Melissa

[skolenik]

I found that paper to be light, to say the least, on survey statistics references. Their simulation was also quite limited in scope, hardly reflective of the complex survey designs used in the real world large scale evaluations. Matching in one package and estimating treatment effect in another package makes it impossible to produce the right standard errors, as they don't account for uncertainty due to externally performed matching. (At any rate, proper design-based survey inference for these problems would be to weight each matched pair by the inverse second-order probability of selection, and the standard errors would involve fourth-order probabilities of selection, which are never available.) So that paper is better than nothing, of course, but should still be treated with a great deal of caution.

Some additional references that DuGoff et.al. should have mentioned (Dave Judkins is a survey statistician who has been involved in designing and implementing such evaluations for the past 10+ years; he just gave a webinar a month or two ago on the topic through the American Statistical Association):

Potter LD, Judkins, D., Piesse A, & Nolin MJ, H. M. (2008). Methodology of the outcome evaluation of the VERB Campaign. American Journal of Preventive Medicine, 34(6S), S230-S240.

Judkins, D., Morganstein, D., Zador, P., Piesse, A., Barrett, B., Mukhopadhyay, P. (2007). Variable Selection and Raking in Propensity Scoring. Statistics in Medicine, 26, 1022-1033.