contrast command and continuous 3-way interaction terms

[Steve Johnson]

Hi!
I've got a question referring the use of continuous by continuous by continuous interaction and the contrast post estimation command:
How do I apply the contrast command in order to test whether the interaction effect of XZ differs significantly between W= -.54 W=-.035 W=.53 (which are the 20th, 50th and 80th percentile of the mean centered variable W)?
The code below leads to the following error message: "interactions without factor variables are not allowed".

 
 reg    Y lc.C##lc.W##lc.Z l.Y, vce(cluster ccode) margins,    dydx(l.X)    at(lc.Z    =(-.4(0.1).4)    lc.W=( -.54 -.035 .53 1.60)) marginsplot                                                                                         contrast    lc.W#lc.X#lc.Z,    nowald    pveffects

Any comments appreciated! Thanks in advance!

[Steve Johnson]

I removed a mistake. hear the correct code:

reg Y lc.C##lc.W##lc.Z l.Y, vce(cluster ccode) margins, dydx(l.X) at(lc.Z =(-.4(0.1).4) lc.W=( -.54 -.035 .53 1.60)) marginsplot contrast lc.W#lc.X#lc.Z, nowald pveffects

[Steve Johnson]

Sorry, for mixing up the line breaks! Once again, any comments to the use of the CONTRAST command appreciated! Thanks a lot in advance!

reg Y lc.X##lc.W##lc.Z l.Y, vce(cluster ccode)

margins, dydx(l.X) at(lc.Z =(-.4(0.1).4) lc.W=( -.54 -.035 .53 1.60))

marginsplot

contrast lc.X#lc.W#lc.Z, nowald pveffect

[Doug Hemken]

Not totally clear what you want to estimate here, but perhaps you want lincom?

[Steve Johnson]

Thanks for your answer!
I want to find out whether the interaction effect of XZ is different for different values of W.
Therefore I first regress Y on X, Z, W (including all the respective combinations of these 3 variables which Stata does automatically applying factor variables ##) and the lagged dependent variable and further controls.
Secondly, I estimate the conditional effect for X over the whole range of Z (from -0.4 to 0.4) and for certain values of W, namely W=( -.54 -.035 .53 1.60).
As a result I graph the 4 marginal effect plots (one for each value of W) in one graph.
However, now I want to know whether the differences between the marginal effect for W= -0.54 and -0.035 is significant, as well as between W= -0.54 and 0.53 ...and so on.
Any ideas?

[Doug Hemken]

So you are asking if
_b[x#z] + (-.54)*_b[x#z#w] - (_b[x#z] + (0.53)*_b[x#z#w]) = 0
that is
(-1.07)*_b[x#z#w] = 0
?

That is the sort of thing lincom will calculate.

[Steve Johnson]

sorry, I don't understand your formula.
What I want to test is the difference between the different regression models (based on the different values of W)

1) _b[x]*X + _b[z]*Z + _b[w]*W + _b[x#z]*XZ + _b[x#w]*XW + _b[w#z]*WZ + (-.54)*_b[x#z#w]

2) _b[x]*X + _b[z]*Z + _b[w]*W + _b[x#z]*XZ + _b[x#w]*XW + _b[w#z]*WZ + (-.035)*_b[x#z#w]

3) _b[x]*X + _b[z]*Z + _b[w]*W + _b[x#z]*XZ + _b[x#w]*XW + _b[w#z]*WZ + (.53)*_b[x#z#w]

Does that help?

[Steve Johnson]

in addition to the above:

1) _b[x]*X + _b[z]*Z + _b[w]*W + _b[x#z]*XZ + _b[x#w]*XW + _b[w#z]*WZ + (-.54)*XZW

2) _b[x]*X + _b[z]*Z + _b[w]*W + _b[x#z]*XZ + _b[x#w]*XW + _b[w#z]*WZ + (-.035)*XZW

3) _b[x]*X + _b[z]*Z + _b[w]*W + _b[x#z]*XZ + _b[x#w]*XW + _b[w#z]*WZ + (.53)*XZW

Estimating the marginal effects dydx in order to graph the marginal effect plots would result in

1) _b[x] + _b[x#z]*Z + _b[x#w]*W + (-.54)*XZW

2) _b[x] + _b[x#z]*Z + _b[x#w]*W + (-.035)*XZW

3) _b[x] + _b[x#z]*Z + _b[x#w]*W + (.53)*XZW

The problem is, how do I specify the contrast command to find out whether these different marginal effect plots differ significantly from each other?

[Jeff Pitblado (StataCorp)]

contrast will not work with terms that lack a factor variable in them.
However, margins has a contrast() option. Based on Steve's
most recent post, I believe the command is.

margins, dydx(l.X) at(lc.W=( -.54 -.035 .53 1.60)) contrast(atcontrast(r._at))

I assume where Steve posted XZW, he meant _b[x#z#w] as in his earlier posts.

[Steve Johnson]

Jeff!
Thanks a lot for your helpful answer!

I assume where Steve posted XZW, he meant _b[x#z#w] as in his earlier posts.

Of course you are right! Apologies for all the couple of mistakes in the posts above, but unfortunately I did not figure out how to edit or even delete my own posts (which is probably not allowed?). So, here is another try:

1) _b[x] + _b[x#z]*Z + _b[x#w]*(-.54) + _b[x#z#w]*Z*(-.54)
2) _b[x] + _b[x#z]*Z + _b[x#w]*(-.035) + _b[x#z#w]*Z*(-.035)
3) _b[x] + _b[x#z]*Z + _b[x#w]*(.53) + _b[x#z#w]*Z*(.53)

contrast will not work with terms that lack a factor variable in them.

This is what I do not understand, because every single predictor in the regression model as well as within the margins-command is written as factor variable (i.e. including the factor operator „c.“ for continuous). At least I don’t see the mistake yet ;-)


Applying Jeff’s solution works fine. However, I´m unsure whether the output addresses the question whether the differences between the marginal effects (XZ) for W= -.54, W=-0.035 and W=0.53 are significantly different from zero, because for all values of W the F-test shows exactly the same values indicating statistical significance at the 10%-level. Is this correct?
And if yes, is the „Contrast Delta-method“ table (below) then of any further meaning?

  Contrasts of average marginal effects
  Model VCE    : Robust
  
  Expression   : Linear prediction, predict()
  dy/dx w.r.t. : L.log_health_aidpc_c
  
  1._at        : L.GOVERNAN~c    =        -.54
  
  2._at        : L.GOVERNAN~c    =       -.035
  
  3._at        : L.GOVERNAN~c    =         .53
  
  4._at        : L.GOVERNAN~c    =         1.6
  
  ------------------------------------------------
               |         df           F        P>F
  -------------+----------------------------------
  L.log_hea~_c |
           _at |
     (2 vs 1)  |          1        3.33     0.0708
     (3 vs 1)  |          1        3.33     0.0708
     (4 vs 1)  |          1        3.33     0.0708
        Joint  |          1        3.33     0.0708
               |
   Denominator |        105
  ------------------------------------------------
  
  ----------------------------------------------------------------------
                       |   Contrast Delta-method
                       |      dy/dx   Std. Err.     [95% Conf. Interval]
  ---------------------+------------------------------------------------
  L.log_health_aidpc_c |
                   _at |
             (2 vs 1)  |  -.0062017   .0033978     -.0129389    .0005354
             (3 vs 1)  |  -.0131403   .0071992     -.0274151    .0011345
             (4 vs 1)  |  -.0262806   .0143985     -.0548301     .002269
  ----------------------------------------------------------------------

Lastly, if W is a dummy variable (applying the operator i.), then using the margins command after the regression works perfectly.

reg    Y    i.W##lc.X##lc.Z
margins,    dydx(l.X)    at(l.Z =(-.4(0.1).4)    W=(0    1))

Thanks a lot in advance!

[Jeff Pitblado (StataCorp)]

Steve replied to my comment: contrast will not work with terms that
lack a factor variable in them.

This is what I do not understand, because every single predictor in the
regression model as well as within the margins-command is written as factor
variable (i.e. including the factor operator „c.“ for continuous).

The varlist of predictors specified in Steve's regress
model is using the interaction operator, and that, along with c.,
are generically identified as factor variables operators; however, they
do not specifically indicate that any of the variables in the model are
factor variables (categorical instead of continuous).

The help file for contrast states the following

contrast termlist ...

where termlist is a list of factor variables or interactions that
appear in the current estimation results. The variables may be typed
with or without contrast operators, and you may use any factor-variable
syntax.

Each term in termlist must identify at least one factor variable in the model,
regardless of which variable operators are specified, otherwise there are no
levels for contrast to difference.

[Jeff Pitblado (StataCorp)]

The following is a longish discussion with an example showing what margins is calculating.

Here is some Stata code that simulates a dataset similar to the one implied
by Steve's model.

set seed 12345

* simulate clusters
set obs 50
gen ccode = _n
gen u = rnormal()

* simulate time within clusters
expand 20
sort ccode
by ccode : gen time = _n
tsset ccode time

* simulate the model variables
gen x = rnormal()
gen w = rnormal()
gen z = rnormal()
by ccode : gen y = - l.x + l.w + lc.x*lc.w + 2*l.z - lc.x*lc.z + .5*lc.w*lc.z - .02*lc.x*lc.w*lc.z + 3 + u + rnormal()

keep ccode time y x w z
keep if y < .

regress y lc.x##lc.w##lc.z l.y, vce(cluster ccode)

Let's compute the predictive marginal effecs of lc.x, with lc.w
fixed at value -.5, 0, and .5.

gen p_at1 = _b[lc.x] + _b[lc.x#lc.w]*(-.5) + _b[lc.x#lc.z]*lc.z + _b[lc.x#lc.w#lc.z]*(-.5)*lc.z
gen p_at2 = _b[lc.x] + _b[lc.x#lc.w]*(0) + _b[lc.x#lc.z]*lc.z + _b[lc.x#lc.w#lc.z]*(0)*lc.z
gen p_at3 = _b[lc.x] + _b[lc.x#lc.w]*(.5) + _b[lc.x#lc.z]*lc.z + _b[lc.x#lc.w#lc.z]*(.5)*lc.z

Here is a summary of these new varaibles

. sum p_at?

    Variable |        Obs        Mean    Std. Dev.       Min        Max
-------------+---------------------------------------------------------
       p_at1 |        900   -1.534162    .9254188  -4.507213   1.141146
       p_at2 |        900   -1.058979    .9333469    -4.0575   1.639248
       p_at3 |        900    -.583796    .9412751  -3.607788   2.137351

Now let's get margins to do this for us.

. margins , dydx(l.x) at(lc.w=(-.5 0 .5))

Average marginal effects                        Number of obs     =        900
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : L.x

1._at        : L.w             =         -.5

2._at        : L.w             =           0

3._at        : L.w             =          .5

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
L.x          |
         _at |
          1  |  -1.534162   .0510119   -30.07   0.000    -1.636674    -1.43165
          2  |  -1.058979   .0439715   -24.08   0.000    -1.147343    -.970615
          3  |   -.583796   .0495915   -11.77   0.000    -.6834537   -.4841382
------------------------------------------------------------------------------

Now we can contrast the predictive marginal effects between our fixed values
of lc.w.

gen p_at2vs1 = p_at2 - p_at1
gen p_at3vs1 = p_at3 - p_at1

Notice that these variables could have been compute directly using

gen p_at2vs1 = _b[lc.x#lc.w]*(.5) + _b[lc.x#lc.w#lc.z]*(.5)*lc.z
gen p_at3vs1 = _b[lc.x#lc.w]*( 1) + _b[lc.x#lc.w#lc.z]*( 1)*lc.z

Which shows us that the values in p_at2vs1 are half the values in
p_at3vs1. This explains why the test statistics are the same.

Here we get margins to compute these contrasted marginal effects for use.

. margins , dydx(l.x) at(lc.w=(-.5 0 .5)) contrast(atcontrast(r._at) nowald pveffect)

Contrasts of average marginal effects
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : L.x

1._at        : L.w             =         -.5

2._at        : L.w             =           0

3._at        : L.w             =          .5

-----------------------------------------------------
             |   Contrast Delta-method
             |      dy/dx   Std. Err.      t    P>|t|
-------------+---------------------------------------
L.x          |
         _at |
   (2 vs 1)  |   .4751831   .0244391    19.44   0.000
   (3 vs 1)  |   .9503661   .0488782    19.44   0.000
-----------------------------------------------------

[Steve Johnson]

Dear Jeff,

thank you very much for clarifying the correct use of the contrast command and factor variables! I appreciate your patience and I'm very glad this expert forum exists! Following your advice I estimate the the regression model and calculate the margins and their contrasts for the simulated dataset.

regress y lc.x##lc.w##lc.z l.y, vce(cluster ccode)
margins , dydx(l.x) at(lc.w=(-.5 0 .5))
margins , dydx(l.x) at(lc.w=(-.5 0 .5)) contrast(atcontrast(r._at) nowald pveffect)
marginsplot

Graphing the marginal effect plots, however, shows the marginal effect dydx (on the y-axis) for different values W (on the x-axis). However, I want to graph the marginal effect dydx (on the y-axis) for different values Z (on the x-axis) and this by different values of W. So that I get one marginal effect plot for each W=-0.5, W=0 and W=0.5, of which I afterwards want to prove whether the difference between these plots is significant. Based on your suggestion I use the margins command in combination with contrast again and include both w and z.

regress y lc.x##lc.w##lc.z l.y, vce(cluster ccode)
margins , dydx(l.x) at(lc.z=(-.4(0.1).4) lc.w=(-.5 0 .5)) contrast(atcontrast(r._at) nowald pveffect)
marginsplot

This provides exactly the graphs I am looking for, BUT by using the contrast operator "r.", the contrast command only provides evidence whether each of the 26 combinations of w and z are significantly different from the reference category, being the minimum of w and z. But I only want to know whether the whole marginal effect plot for W=-0.5 differs significantly from the whole marginal effect plot for W=0.5 (or W=0) (Changing the contrast operator into "a." or "ar." shows strange results).
Is this possible?

As already mentioned, so far, I was only able to do this if w is a dummy variable (see below): - correct me if I am wrong- here the (seperate) contrast command provides evidence whether the difference between the marginal effect plot for W=0 and the plot for W=1 is significant, which is not the case in your simulated dataset P>(t)=0.648.

First w is transformed into a dummy variable w0 and included into the regression model, the margins command provides the derivative for all the combinations of z and w0 (9x2=18 conditional marginal effects). The slopes at some of these combinations of w and z are significantly different from zero, others not. Plotting the derivates results in two average marginal effect plots for W=0 and W=1. The contrasts of marginal linear predictions however reveal that the difference is not significant. This is what I want to repeat for w=-0.5, w=0 and w=0.5.

gen w0=w
replace w0=0 if w<0
replace w0=1 if w>0
            
reg    y    lc.x##li.w0##lc.z, vce(cluster    ccode)
margins,    dydx(l.x)    at(lc.z    =(-.4(0.1).4)    l.w0=(0    1))
marginsplot
contrast    lc.x#li.w0#lc.z,    nowald    pveffects

... so, against this background: can I somehow apply the same procedure for the three different values of a continious variable w and determine whether the marginal effect dydx (y-axis) for different values Z (x-axis) differs signficantly between W=-0.5, W=0 and W=0.5?

Thanks a lot in advance for further comments!

[Jeff Pitblado (StataCorp)]

OK, with so many values of lc.z it's too easy to get lost, so I will pair it down to 3 values.

. margins , dydx(l.x) at(lc.z=(-.4 0 .4) lc.w=(-.5 0 .5))

Average marginal effects                        Number of obs     =        900
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : L.x

1._at        : L.w             =         -.5
               L.z             =         -.4

2._at        : L.w             =         -.5
               L.z             =           0

3._at        : L.w             =         -.5
               L.z             =          .4

4._at        : L.w             =           0
               L.z             =         -.4

5._at        : L.w             =           0
               L.z             =           0

6._at        : L.w             =           0
               L.z             =          .4

7._at        : L.w             =          .5
               L.z             =         -.4

8._at        : L.w             =          .5
               L.z             =           0

9._at        : L.w             =          .5
               L.z             =          .4

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
L.x          |
         _at |
          1  |  -1.126802   .0614188   -18.35   0.000    -1.250227   -1.003376
          2  |  -1.501418   .0515804   -29.11   0.000    -1.605073   -1.397763
          3  |  -1.876034   .0486431   -38.57   0.000    -1.973786   -1.778283
          4  |  -.6481288    .050339   -12.88   0.000    -.7492888   -.5469688
          5  |  -1.025955    .044238   -23.19   0.000    -1.114854    -.937055
          6  |   -1.40378   .0441206   -31.82   0.000    -1.492444   -1.315117
          7  |  -.1694558   .0526904    -3.22   0.002    -.2753412   -.0635705
          8  |  -.5504909   .0495775   -11.10   0.000    -.6501206   -.4508612
          9  |   -.931526   .0524809   -17.75   0.000     -1.03699   -.8260617
------------------------------------------------------------------------------

. marginsplot

Notice that the margins plots are parallel lines in lc.z. If I understand Steve, he would like to
compare the plots between levels of lc.w but within the levels of lc.z. If we post the above
margins results, then we can use custom contrasts. Here is an example comparing the
bottom line to the middle one.

The custom contrast {_at -1 0 0 1} compares the first and fourth marginal effects, that is,
it compares the marginal effect of lc.x when lc.w=-.5 and lc.z=-.4
to the marginal effect of lc.x when lc.w=0 and lc.z=-.4. The next custom contrast
shifts the value of lc.z to 0, holding all the other things fixed. The last custom contrast
shifts the value of lc.z to .4.

. margins , dydx(l.x) at(lc.z=(-.4 0 .4) lc.w=(-.5 0 .5)) post
(output omitted)

. contrast {_at -1 0 0 1} {_at 0 -1 0 0 1} {_at 0 0 -1 0 0 1}, pveffects

Contrasts of average marginal effects
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : L.x

1._at        : L.w             =         -.5
               L.z             =         -.4

2._at        : L.w             =         -.5
               L.z             =           0

3._at        : L.w             =         -.5
               L.z             =          .4

4._at        : L.w             =           0
               L.z             =         -.4

5._at        : L.w             =           0
               L.z             =           0

6._at        : L.w             =           0
               L.z             =          .4

7._at        : L.w             =          .5
               L.z             =         -.4

8._at        : L.w             =          .5
               L.z             =           0

9._at        : L.w             =          .5
               L.z             =          .4

------------------------------------------------
             |         df           F        P>F
-------------+----------------------------------
L.x          |
         _at |
        (1)  |          1      309.52     0.0000
        (2)  |          1      375.38     0.0000
        (3)  |          1      363.48     0.0000
      Joint  |          2      190.94     0.0000
             |
 Denominator |         49
------------------------------------------------

-----------------------------------------------------
             |   Contrast Delta-method
             |      dy/dx   Std. Err.      t    P>|t|
-------------+---------------------------------------
L.x          |
         _at |
        (1)  |    .478673   .0272077    17.59   0.000
        (2)  |   .4754636   .0245405    19.37   0.000
        (3)  |   .4722542   .0247704    19.07   0.000
-----------------------------------------------------

Here is an example comparing the bottom line to the top one.

. contrast {_at -1 0 0 0 0 0 1} {_at 0 -1 0 0 0 0 0 1} {_at 0 0 -1 0 0 0 0 0 1}
> , pveffects

Contrasts of average marginal effects
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : L.x

1._at        : L.w             =         -.5
               L.z             =         -.4

2._at        : L.w             =         -.5
               L.z             =           0

3._at        : L.w             =         -.5
               L.z             =          .4

4._at        : L.w             =           0
               L.z             =         -.4

5._at        : L.w             =           0
               L.z             =           0

6._at        : L.w             =           0
               L.z             =          .4

7._at        : L.w             =          .5
               L.z             =         -.4 [b]lc.w[b]=-.5 and lc.z=-.4

8._at        : L.w             =          .5
               L.z             =           0

9._at        : L.w             =          .5
               L.z             =          .4

------------------------------------------------
             |         df           F        P>F
-------------+----------------------------------
L.x          |
         _at |
        (1)  |          1      309.52     0.0000
        (2)  |          1      375.38     0.0000
        (3)  |          1      363.48     0.0000
      Joint  |          2      190.94     0.0000
             |
 Denominator |         49
------------------------------------------------

-----------------------------------------------------
             |   Contrast Delta-method
             |      dy/dx   Std. Err.      t    P>|t|
-------------+---------------------------------------
L.x          |
         _at |
        (1)  |   .9573459   .0544154    17.59   0.000
        (2)  |   .9509272    .049081    19.37   0.000
        (3)  |   .9445084   .0495408    19.07   0.000
-----------------------------------------------------

It should be reassuring that the test statistics yield the same values between these two
calls to contrast, since the margin prediction plots were parallel.

[Steve Johnson]

This is exactly what I was looking for! Excellent! Thank you!

One last question:
Applying the custom contrast command to my dataset, in which the 3 marginal effect plots (for W= -0.54; W= -0.035; W=0.53) are not parallel, the test statistics are identical.

contrast {_at -1 0 0 1} {_at 0 -1 0 0 1} {_at 0 0 -1 0 0 1} {_at 0 0 0 -1 0 0 1} {_at 0 0 0 0 -1 0 0 1} {_at 0 0 0 0 0 -1 0 0 1} {_at -1 0 0 0 0 0 1} {_at 0 -1 0 0 0 0 0 1} {_at 0 0 -1 0 0 0 0 0 1}, pveffects

-------------------------------------------------------------
                     |   Contrast Delta-method
                     |      dy/dx   Std. Err.      t    P>|t|
---------------------+---------------------------------------
L.log_health_aidpc_c |
                 _at |
                (1)  |   -.025324   .0139647    -1.81   0.073
                (2)  |  -.0060508   .0033767    -1.79   0.076
                (3)  |   .0132225   .0126929     1.04   0.300 

                (4)  |  -.0283328   .0156239    -1.81   0.073
                (5)  |  -.0067697   .0037779    -1.79   0.076
                (6)  |   .0147935   .0142009     1.04   0.300 

                (7)  |  -.0536569   .0295886    -1.81   0.073
                (8)  |  -.0128205   .0071545    -1.79   0.076
                (9)  |    .028016   .0268938     1.04   0.300 
-------------------------------------------------------------

Is that something I should worry about?