Moderation: Insignificant interaction term but significant marginal effects

[Robin Busse]

Dear Statalisters,

I have a cross-sectional dataset (n = 1,145). The theory that I´m using suggests that Z moderates the relationship between X and Y. To investigate this moderation, I am estimating a linear regression model (OLS) with an interaction between X and Z. Both the independent variable (X) and the moderator variable (Z) are continuous. It confuses me that the coefficient of the interaction term is insignificant (p=0.147), whereas the marginal effects of X on Y are significant at each of the values of the moderator (Z).

In line with the theory, the results of the marginal effects show that the effect of X on Y decreases as Z increases. Can I assume that Z moderates the relationship between X and Y even though the interaction term is not significant?

. regress Y c.X##c.Z

      Source |       SS           df       MS      Number of obs   =     1,145
-------------+----------------------------------   F(3, 1141)      =    108.86
       Model |  115.453988         3  38.4846625   Prob > F        =    0.0000
    Residual |  403.380401     1,141  .353532341   R-squared       =    0.2225
-------------+----------------------------------   Adj R-squared   =    0.2205
       Total |  518.834389     1,144  .453526564   Root MSE        =    .59459

------------------------------------------------------------------------------
           Y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           X |   .2563362   .1232238     2.08   0.038     .0145655     .498107
           Z |  -.3460246   .0897433    -3.86   0.000    -.5221049   -.1699442
             |
     c.X#c.Z |  -.0499119   .0344294    -1.45   0.147    -.1174639    .0176402
             |
       _cons |   2.711265   .3254729     8.33   0.000     2.072673    3.349858
------------------------------------------------------------------------------

. margins, at(Z=(1(0.2)4)) dydx(X)

Average marginal effects                        Number of obs     =      1,145
Model VCE    : OLS

Expression   : Linear prediction, predict()
dy/dx w.r.t. : X

1._at        : Z               =           1

2._at        : Z               =         1.2

3._at        : Z               =         1.4

4._at        : Z               =         1.6

5._at        : Z               =         1.8

6._at        : Z               =           2

7._at        : Z               =         2.2

8._at        : Z               =         2.4

9._at        : Z               =         2.6

10._at       : Z               =         2.8

11._at       : Z               =           3

12._at       : Z               =         3.2

13._at       : Z               =         3.4

14._at       : Z               =         3.6

15._at       : Z               =         3.8

16._at       : Z               =           4

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
X            |
         _at |
          1  |   .2064244   .0896159     2.30   0.021     .0305939    .3822548
          2  |    .196442   .0829752     2.37   0.018      .033641    .3592431
          3  |   .1864596   .0763779     2.44   0.015     .0366028    .3363165
          4  |   .1764773   .0698363     2.53   0.012     .0394552    .3134993
          5  |   .1664949   .0633678     2.63   0.009     .0421644    .2908254
          6  |   .1565125   .0569971     2.75   0.006     .0446816    .2683434
          7  |   .1465301   .0507612     2.89   0.004     .0469344    .2461258
          8  |   .1365478   .0447163     3.05   0.002     .0488123    .2242832
          9  |   .1265654   .0389517     3.25   0.001     .0501404    .2029904
         10  |    .116583   .0336118     3.47   0.001     .0506352    .1825309
         11  |   .1066007   .0289327     3.68   0.000     .0498335    .1633678
         12  |   .0966183   .0252839     3.82   0.000     .0470101    .1462265
         13  |   .0866359   .0231578     3.74   0.000     .0411992    .1320726
         14  |   .0766535   .0229809     3.34   0.001     .0315639    .1217432
         15  |   .0666712    .024795     2.69   0.007     .0180222    .1153201
         16  |   .0566888   .0282186     2.01   0.045     .0013225     .112055
------------------------------------------------------------------------------

Here are some further details on the variables I used.

. sum X Z

    Variable |        Obs        Mean    Std. Dev.       Min        Max
-------------+---------------------------------------------------------
           X |      1,145    2.288355    .7850464          1          4
           Z |      1,145    3.453057    .6356878          1          4

Looking forward for any comments and suggestions,
Robin

[Maarten Buis]

No, the p-values in your margins output test the hypothesis that the effects of x is equal to 0 for different values of z. Moderation means that the effect of x changes when z changes. The p-value next to the interaction effect tests the hypothesis that this change in effect of x when z changrs is equal to 0. Your results indicate that you cannot reject that hypothesis.

[Joro Kolev]

Another way to see what Maarten is saying is to take the derivative manually, and your derivative here is

d(EY)/dX = .2563362 -.0499119*Z

The regression output is telling you whether the estimate -.0499119 is significantly different from 0.

The margins command is telling you whether the estimate .2563362 -.0499119*Z is significantly different from 0.

As for the conclusion of Maarten I am not quite agreed: you re not able to reject that the estimate -.0499119 is 0 at the 10% significance level. You can reject this hypothesis at the 14.7% significance level. In other words us not being able to reject that something is 0 at a common significance level such as 1%, 5% or 10% does not mean that this thing is 0.

In your case the key is whether the decrease in the derivative d(EY)/dX that you observe over the range of Z, which decrease is from .206 down to 0.057 is meaningful/substantial, or not.

In your case, I generally keep interactions that theory suggest should be included.

[Maarten Buis]

I was careful not to claim that the interaction is zero. I would say that the data does not contain enough information to decide one way or another. Absence of evidence is not the same as evidence of absence.