Comparing two separate t-tests

[Surya Singh]

Hi all,
I'm trying to compare two differences from two separate t-tests and whether those differences are statistically different. Perhaps better explained by an example:

. ttest absdays_totalcombined_c if treatedpost_control == 0 , by(gender)

Two-sample t test with equal variances
------------------------------------------------------------------------------
   Group |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
    male |     693    14.21356    .6848489    18.02857    12.86893     15.5582
  female |     698    34.59742    1.384127    36.56821    31.87986    37.31498
---------+--------------------------------------------------------------------
combined |   1,391    24.44213    .8204359    30.59907     22.8327    26.05155
---------+--------------------------------------------------------------------
    diff |           -20.38386    1.547674               -23.41989   -17.34783
------------------------------------------------------------------------------
    diff = mean(male) - mean(female)                              t = -13.1706
Ho: diff = 0                                     degrees of freedom =     1389

    Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
 Pr(T < t) = 0.0000         Pr(|T| > |t|) = 0.0000          Pr(T > t) = 1.0000

. ttest absdays_totalcombined_c if treatedpost_control == 1 , by(gender)

Two-sample t test with equal variances
------------------------------------------------------------------------------
   Group |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
    male |     168    16.28571    1.442317    18.69456    13.43819    19.13324
  female |     190    26.82105    2.321296    31.99685    22.24208    31.40003
---------+--------------------------------------------------------------------
combined |     358    21.87709    1.431116    27.07799    19.06262    24.69157
---------+--------------------------------------------------------------------
    diff |           -10.53534    2.816868               -16.07513   -4.995544
------------------------------------------------------------------------------
    diff = mean(male) - mean(female)                              t =  -3.7401
Ho: diff = 0                                     degrees of freedom =      356

    Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
 Pr(T < t) = 0.0001         Pr(|T| > |t|) = 0.0002          Pr(T > t) = 0.9999

So I want to see if the first difference between males and females, -20.38386 is statistically difference from the second difference -10.5354.
How would I go about doing this?

Any advice would be great, thanks!

[Carlo Lazzaro]

Surya:
I would leave -ttest- and switch to -regress- with interaction:

reg absdays_totalcombined_c i.treatedpost_control##i.gender

[Surya Singh]

Surya:
I would leave -ttest- and switch to -regress- with interaction:

reg absdays_totalcombined_c i.treatedpost_control##i.gender

Thanks Carlo! That gave me what I wanted!
Just out of curiosity though, would there be a way to do the above with the t-test command rather than switching to reg? or is it best to switch to reg?

Best,
Surya

[Carlo Lazzaro]

Surya:
I would not sponsor your approach. Following your strategy, you shoud perform -ttest- on the difference of -absdays_totalcombined_c- before and after the treatment; it boils down to a "mirror study" design, in which each id acts as herself/himself control.
Perhaps somethng along the following lines is what you're looking for:

. use http://www.stata-press.com/data/r14/fuel

. g id=_n

. order id , first

. stack mpg1 mpg2, into(mpg) clear

. g id=_n in 1/12

. replace id=_n-12 in 13/24

. g additive=0 in 1/12

. replace additive=1 if additive==.

. label define additive 0 "Old_gas" 1 "New_gas"

. label val additive additive

. sort id

. g race_team=0 in 1/12

. replace race_team=1 in 13/24

. label define race_team 0 "Red" 1 "Black"

. label val race_team race_team

. ttest mpg if additive == 0 , by( race_team )

. ttest mpg if additive == 1 , by( race_team )

.  bysort id : g mpg_diff=mpg[2]-mpg[1]

. bysort id: keep if _n==1

. ttest mpg_diff , by( race_team )

As you can see, the take-home message is: switch to regression (with intearction, in your case).

[Bruce Weaver]

Hi Surya. If you have access to the raw data, then I agree with Carlo that a regression model that includes an interaction term is the most sensible approach. On the other hand, one may only have summary data from two analyses performed by someone else. With those circumstances in mind, my co-author Karl Wuensch and I wrote SPSS and SAS code to compare two linear regression coefficients that come from two independent models. See file # 4 on one of the following pages:

• SPSS syntax files
• SAS code

Bearing in mind that the pooled variances version of the independent groups t-test is equivalent to a linear regression model with a dichotomous predictor, that code could be adapted to do what you want, I expect. But as you can see, it is more onerous that using regression with an interaction term when you have the raw data.

HTH.