How to test if the difference between two groups are significantly different from the difference between two other groups.

[Maiken Munk Jensen]

Dear all,

I am doing an interaction analysis with two factorial variables (informationtype and gender) on attitudes towards childhood vaccination (scaled from 0 - 1).
The variable informationtype has two categories: control and experimental treatment
The gender variable also has two categories: men and women.

My interaction analyses shows:
- There is a significant difference between men and women in my control group (p < 0,05)
- There is an insignificant difference between men and women in my treatment group
- There is a significant difference between men in control and treatment (p < 0,001)
- There is a significant difference between women in control and treatment ( p < 0,001).

My question is: How do I show whether the difference between men and women in the control group is significantly different from the difference between men and women in the treatment group? I.e. whether the difference between the two differences is statistical significant.
According to my output the difference between men and women in control is 4 percent points while the difference in the treatment is -7,8 percent points.


My interaction analysis:


My margins output:



I am using Stata 13.1

I hope someone in here can help me with my frustrations.

Best regards
Maiken Munk

[Carlo Lazzaro]

Maiken:
welcome to this forum.
I do hope that the following toy-example can help:

. sysuse auto.dta
(1978 Automobile Data)

. logistic foreign c.trunk##i.rep78
note: 1.rep78 != 0 predicts failure perfectly
      1.rep78 dropped and 2 obs not used

note: 2.rep78 != 0 predicts failure perfectly
      2.rep78 dropped and 8 obs not used

note: 5.rep78 omitted because of collinearity
note: 2.rep78#c.trunk omitted because of collinearity
note: 5.rep78#c.trunk omitted because of collinearity

Logistic regression                             Number of obs     =         59
                                                LR chi2(5)        =      33.95
                                                Prob > chi2       =     0.0000
Log likelihood = -21.435048                     Pseudo R2         =     0.4420

-------------------------------------------------------------------------------
      foreign | Odds Ratio   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
        trunk |   1.614523   .7188064     1.08   0.282       .67465     3.86376
              |
        rep78 |
           1  |          1  (empty)
           2  |          1  (empty)
           3  |   322.0895   1713.151     1.09   0.278      .009562    1.08e+07
           4  |    2603.57   12716.19     1.61   0.107     .1812117    3.74e+07
           5  |          1  (omitted)
              |
rep78#c.trunk |
           1  |          1  (empty)
           2  |          1  (empty)
           3  |   .4502897   .2243771    -1.60   0.109     .1695675    1.195753
           4  |   .4504883   .2103481    -1.71   0.088     .1803969    1.124963
           5  |          1  (omitted)
              |
        _cons |   .0285479    .127197    -0.80   0.425     4.60e-06    177.0766
-------------------------------------------------------------------------------
Note: _cons estimates baseline odds.

. mat list e(b)

e(b)[1,12]
       foreign:    foreign:    foreign:    foreign:    foreign:    foreign:    foreign:    foreign:    foreign:
                        1b.         2o.          3.          4.         5o.   1b.rep78#   2o.rep78#    3.rep78#
         trunk       rep78       rep78       rep78       rep78       rep78    co.trunk    co.trunk     c.trunk
y1   .47903979           0           0   5.7748293    7.864639           0           0           0  -.79786403

       foreign:    foreign:    foreign:
       4.rep78#   5o.rep78#           
       c.trunk    co.trunk       _cons
y1  -.79742312           0  -3.5561733

. test 2o.rep78#c.trunk=4.rep78#c.trunk

 ( 1)  [foreign]2o.rep78#co.trunk - [foreign]4.rep78#c.trunk = 0

           chi2(  1) =    2.92
         Prob > chi2 =    0.0877

.

[Maiken Munk Jensen]

Dear Carlo,

Thanks for the example!
I don't think it is usable for my problem as I'm not doing a logistic regression, but a standard OLS.

My y is attitudes towards childhood vaccination, which is range scaled from 0 to 1, with high values reflecting positive attitudes.
My x is an experimental treatment.
The interaction variable is gender.

I'm interested in figuring out whether gender interacts with the effects of my treatment. Which I somewhat can confirm with the output above

I want to move beyond the standard interaction model and test whether the difference between men and women in the control group is significantly different from the difference between men and women in the treatment group. I just cannot figure out how to do this in Stata.

I have circled the differences which I'm interested in testing.



Something tells me that it's a simple t-test, but I don't know how to do that on the differences.

[Carlo Lazzaro]

Maiken:
still from -auto.dta-:

. reg price i.foreign##i.rep78
note: 1.foreign#1b.rep78 identifies no observations in the sample
note: 1.foreign#2.rep78 identifies no observations in the sample
note: 1.foreign#5.rep78 omitted because of collinearity

      Source |       SS           df       MS      Number of obs   =        69
-------------+----------------------------------   F(7, 61)        =      0.39
       Model |    24684607         7  3526372.43   Prob > F        =    0.9049
    Residual |   552112352        61  9051022.16   R-squared       =    0.0428
-------------+----------------------------------   Adj R-squared   =   -0.0670
       Total |   576796959        68  8482308.22   Root MSE        =    3008.5

-------------------------------------------------------------------------------
        price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
      foreign |
     Foreign  |   2088.167   2351.846     0.89   0.378     -2614.64    6790.974
              |
        rep78 |
           2  |   1403.125   2378.422     0.59   0.557    -3352.823    6159.073
           3  |   2042.574   2204.707     0.93   0.358    -2366.011    6451.159
           4  |   1317.056   2351.846     0.56   0.578    -3385.751    6019.863
           5  |       -360   3008.492    -0.12   0.905    -6375.851    5655.851
              |
foreign#rep78 |
   Foreign#1  |          0  (empty)
   Foreign#2  |          0  (empty)
   Foreign#3  |  -3866.574   2980.505    -1.30   0.199    -9826.462    2093.314
   Foreign#4  |  -1708.278   2746.365    -0.62   0.536    -7199.973    3783.418
   Foreign#5  |          0  (omitted)
              |
        _cons |     4564.5   2127.325     2.15   0.036      310.651    8818.349
-------------------------------------------------------------------------------

. test 1.foreign#3.rep78=1.foreign#4.rep78

 ( 1)  1.foreign#3.rep78 - 1.foreign#4.rep78 = 0

       F(  1,    61) =    0.87
            Prob > F =    0.3550