Weighted (-svy- command) paired sample t test

[Corey Bryant]

Hello all,

I tried searching this forum and the general internet but could not find what I was looking for. Also, if I am missing anything, I apologize in advance.

Quick information about the data and analytic plan
I have a data set that is survey data that we now have 2 years of data for (years 2016 and 2017); the analytic plan is to test for a difference between 2016 and 2017 on a continuous variable (days missed work).

The issue
the -ttest- command doesn't support -svy-, nor -pweights-.

Things I have looked into thus far
I have tried

 svy: mean variable_of_interest, over(year)

with a followup of either -lincom- or -test-, but these commands don't account for/allow for paired samples. I have also looked into the -anova- command, but that doesn't allow for -pweights-; also the -oneway- command doesn't allow for -pweights- either. I'm currently looking into the possibility of using the -regress- command since it allows for -svy-, but am unsure how to take into account the repeated measure/paired sample design.

Any thoughts and/or help would be appreciated; thank you for your time.

Best,

Corey

[Rich Goldstein]

use -regress-; first, generate a new variable that is the difference; then that new variable is your outcome variable and you can use pweights with -regress; no predictors/covariates; to convince yourself that this is the same for unweighted data; use paired t-test and regress without weights

[Dave Airey]

You can reproduce the paired ttest with regress:

clear
use http://www.stata-press.com/data/r15/fuel
describe
ttest mpg1 = mpg2
generate car = _n
list
reshape long mpg, i(car) j(treatment)
regress mpg i.treatment i.car

[Corey Bryant]

I tested both of them; using -regress- with just the difference between the years as the outcome produces an identical output to the -ttest- command (besides the extra information given from the -regress- command). This will be the method I use, thank you both!

[Steve Samuels]

Be sure to use svy: regress.

However a test of hypothesis is probably not appropriate for your problem. You are attempting to test the null hypothesis that the average difference in lost work days is exactly zero (i.e. the means are identical). However in the specific population in those years, the null hypothesis cannot be true, except by shear chance. (Imagine a population of 3,145,267 people and that you ascertain lost work days in each of them in each year. Do you think that the mean of that difference will be exactly zero?).

The question is, then, how different were the means, and that question is answered by a confidence interval. Generally (there are exceptions), tests of hypotheses are appropriate only for causal questions, those in which you want to extrapolate beyond the specific population at the times of the survey.

If a specific policy to affect lost work days was introduced in the second year and you'd like to ask if that would happen in other settings then you can make a case for a test of a causal hypothesis. . See related references this thread , which explains why for testing of a causal null hypothesis, the finite correction should never be used.

[Steve Samuels]

The first sentence of the last paragraph has a typo. It should have been:

If a specific policy to affect lost work days was introduced in the second year and you'd like to ask what would happen in other settings, then you can make a case for a test of a causal hypothesis

.