Interaction effect in multiply regression: the simple effect is non-significant but the interaction effect is significant

[David Wong]

Dear Stata Statisticians,

I have one unbalanced panel data, including 1,000 households and 8,790 individuals. T= 9. The simple model specification is:

Yit = a + bXit + cZit + dC_Xit*C_Zit

Yit is the dependent variable of individual i in period t, which is continuous;
Xit is the independent variable of individual i in period t, which is continuous;
Zit is the moderator of individual i in period t, which is also continuous;
C_Xit*C_Zit is the interaction item of centered Xit and centered Zit.

I use command: xtreg Yit Xit Zit c.C_Xit#c.C_Zit, fe r (fixed effect linear model)

The results that I have now is:
b is non-significant;
c is significant;
d is significant.

According to Jaccard and Turrisi (2003), coefficients b is non-significant, indicating that there is no simple effect of Xit on Yit. However, coefficient d is significant, indicating that Zit moderates the relationship between Xit and Yit. In other words, there is no simple effect but there is interaction effect.

My question is 1) what do these results mean?
2) If there is any statistical value of these results, how should I discuss the case that no simple effect but having interaction effect?

I am still in early stage of my research, please verify me if I say something wrong. Any suggestion of related references will be really appreciated. Thank you so much for your kind help.

Best regards,
David


Reference:
Jaccard, J., & Turrisi, R. (2003). Interaction effects in multiple regression (No. 72). Sage.

[Clyde Schechter]

According to Jaccard and Turrisi (2003), coefficients b is non-significant, indicating that there is no simple effect of Xit on Yit. However, coefficient d is significant, indicating that Zit moderates the relationship between Xit and Yit. In other words, there is no simple effect but there is interaction effect.

I am not familiar with the reference you site, and it may be that you are interpreting their words differently than I would. But what you say in this sentence is flat-out wrong.

In the model you describe, there is no such thing as a simple effect of X. This is an interaction model, so by using that model you are stipulating that the effect of X depends on the value of Z. The coefficient b in that model is the effect of X conditional on Z = 0. Whether that coefficient is of any interest depends on whether 0 is an interesting, or even possible value of Z. Similarly the coefficient c is the effect of Z conditional on X = 0. It, too, may or may not be of any interest, depending on whether 0 is an interesting, or even possible value of X.

The way to understand interaction models is to follow them up with an appropriate -margins- command, and usually with -marginsplot- as well. The first step is to pick interesting variables of X and Z. These could be values that are particularly important in your context, or, could just be a set of values that span the range of X and Z observed in your data. For purposes of illustration, I will assume that interesting values of X are 1, 2, 3, 4, and 5, and that interesting values of Z are 20, 40, 60, and 80.

So after running your regression command, run:

margins, dydx(X) at (Z = (20 40 60 80))
marginsplot
margins, dydx(Z) at (X = (1 2 3 4 5))

This will give you tables and graphs of the marginal effects of X and Z at the interesting values of the other. It is these results that you should focus on, not the coefficients in the regression output.

The closest I can think of to make some kind of sense out of what you attribute to Jaccard and Turrisi is if X and Z are both centered around zero, and by "simple effect" they mean effect conditional on the other variable taking its mean value. Otherwise it is just nonsense. If you have truly interpreted Jaccard and Turrisi correctly, then I suggest you burn the article and do your best to forget you ever read it.

[Paul T Seed]

I'm all in favour of making a bonfire of bad papers; but in this instance I would recommend drawing some graphs. The most likely explanation is that under some circumstances , Y rises as X rises; and under other circumstances it falls. Clyde's code will produce the relevant summaries, and even some graphs to look at.

[David Wong]

I am not familiar with the reference you site, and it may be that you are interpreting their words differently than I would. But what you say in this sentence is flat-out wrong.

In the model you describe, there is no such thing as a simple effect of X. This is an interaction model, so by using that model you are stipulating that the effect of X depends on the value of Z. The coefficient b in that model is the effect of X conditional on Z = 0. Whether that coefficient is of any interest depends on whether 0 is an interesting, or even possible value of Z. Similarly the coefficient c is the effect of Z conditional on X = 0. It, too, may or may not be of any interest, depending on whether 0 is an interesting, or even possible value of X.

The way to understand interaction models is to follow them up with an appropriate -margins- command, and usually with -marginsplot- as well. The first step is to pick interesting variables of X and Z. These could be values that are particularly important in your context, or, could just be a set of values that span the range of X and Z observed in your data. For purposes of illustration, I will assume that interesting values of X are 1, 2, 3, 4, and 5, and that interesting values of Z are 20, 40, 60, and 80.

So after running your regression command, run:

margins, dydx(X) at (Z = (20 40 60 80))
marginsplot
margins, dydx(Z) at (X = (1 2 3 4 5))

This will give you tables and graphs of the marginal effects of X and Z at the interesting values of the other. It is these results that you should focus on, not the coefficients in the regression output.

The closest I can think of to make some kind of sense out of what you attribute to Jaccard and Turrisi is if X and Z are both centered around zero, and by "simple effect" they mean effect conditional on the other variable taking its mean value. Otherwise it is just nonsense. If you have truly interpreted Jaccard and Turrisi correctly, then I suggest you burn the article and do your best to forget you ever read it.

Dear Clyde,

Thank you for your reply and verify me. After further reading the documents that I have, "simple effect" is the effect conditional on the other taking its mean value. Therefore, Jaccard and Turrisi (2003) and you are both correct, in terms of explaining and exploiting the interaction effect by

 margins

and

 marginsplot

. I have a misunderstand regarding how to do so and I apology for this inconvenience.

I also find that Mehmetoglu and Jakobsen (2016) particularly explains the approaches to analyze the interaction effect in Stata. I will have a look at this one.

Best regards,
David

Reference:
Mehmetoglu, M., & Jakobsen, T. G. (2016). Applied Statistics Using Stata: A Guide for the Social Sciences. SAGE.

[David Wong]

I'm all in favour of making a bonfire of bad papers; but in this instance I would recommend drawing some graphs. The most likely explanation is that under some circumstances , Y rises as X rises; and under other circumstances it falls. Clyde's code will produce the relevant summaries, and even some graphs to look at.

Dear Paul,

Thank you for your reply and good suggestion. I will take time to read more related articles.

Best regards,
David