Calculate power with weighted sample

[John Havens]

Hi,
I want to calculate the power of my sample using weight from probabilistic survey design. The -power- in Stata doesn't have any option for weight or survey design. My approach is just to calculate power using the unweighted sample, and guess the power in the weighted sample would be smaller than what I get from the unweighted sample. Can anyone give me an input?
Thanks

[John Havens]

Hi,
Has anyone calculated power from a unweighted sample and defended its use for weighted sample?
I want to know at least it is not too silly to think about that approach.
Thanks

[Dick Campbell]

It's hard to answer this without knowing more about the weights and the survey design. In general, specifying a full survey design including psu's and strata will increase standard errors for descriptive statistics such as means, often by as much as a third. So, you could estimate the design effect (the influence of the sample design) on SE's) and adjust your power analysis to account for it. However, if you are doing something even a bit more complex, say an OLS regression model, it's hard to give advice without more information.

[John Havens]

Dear Professor Campbell,
I'm sorry for the lack of information.
I use NHANES data, which was sampled by multiple stage sampling with PSU and Strata.
I intend to run a logistic regression to see the effect of exposure on cardiovascular disease. The prevalence of CVD in the unexposed group is 8.6, and that in the exposed group is 10.5. My unweighted sample is 2334, weighted sample is around 89 million. My type 1 error should be 5%. I will have around 1 or 2 confounders in my model (which would decrease the power further), but i just need to calculate the power for the crude model.
Thank you

[Dick Campbell]

Just a few more points here. First, many people would argue that power analysis is moot in this cse. You have secondary data and the sample size is what it is. You have no control over this and so your ability to detect the effect that you discussed in your last post is fixed. Secondly, yes, the weights distributed with NHANES give you descriptive estimates of population totals, but the estimated standard errors in your logistic regression will be based on the actual sample size after correcting for the design effect associated with the complex sample. Finally, if you are using a subsample of the data, your survey analysis must reflect that. There are several examples of how to do that in the SVY manual.

[John Havens]

Thank you very much!

[John Havens]

Just for if someone is also looking for the same question. I got an answer from CDC; they said that they did posterior power calculation by using an effective sample size, which was calculated by the unweighted sample divided by the design effect. The design effect needs to be calculated for each outcome.