When including the moderator, my main relationship gets insignificant. Still okay?

[Niklas Max]

Hello to everyone,

as I'm currently on my Master's Thesis, I use Stata for the first time. Now, I figured out my variables and included them in a model.
In the first model, where I test the main relationship, I included IV, DV, and seven controls, and run a fractional logit regression, where I found significance.
However, when I include the moderating variable and the interaction term, the main relationship appears to be insignificant.
Can I now still indicate that the main relationship is significant? Or do I have to look at it at the end, when all variables including the moderators are in there.

Thank you for your help in advance!!

Niklas

[Clyde Schechter]

The key thing you need to understand is that your "main effect" in the interaction model means something different from the "main effect" in the first model (without interaction).

In a modl containing both DV and DV#M (where M is the variable you interact with DV), the coefficient of DV is not telling you the "main effect" of DV. It is telling you the effect of DV conditional on M = 0. Indeed, when you use an interaction model, you are positing that there is no such thing as the effect of DV. In an interaction model, DV has a different effect corresponding to each value of M. The DV#M coefficient tells you how much the effect of DV conditional on a given value of M differs from the effect of DV conditional on M = 0.

The simplest case is when M is just a 0/1 indicator variable. Then DV has two effects: one when M = 0 (given by the coefficient of DV) and another when M = 1, whose value would be the sum of the DV and DV#M coefficients. If M is a continuous variable, then the effect of DV conditional on the value of M is given by the coefficient of DV + M*coefficient of DV#M.

So what your results tell you is: 1. If you ignore the interaction of DV with M, you will find that the overall effect of DV on IV is "significant." 2. But when the interaction with M is recognized you find that the effect of DV is not significant when M = 0. It may nevertheless be "significant" for M = 1 (or, if M is continuous, for some other important values of M). You can use -lincom- to get those effect estimates. Or, better still, if you used factor-variable notation in your regression command, use -margins-.

In future posts, it is better to show the actual code you used than to describe your models in words. I could have given a much more concise, clear response to your question had you done that, and could have offered specific code to use for calculating the effect(s) of DM conditional on non-zero value(s) of M.

[Richard Williams]

This handout explains why the interpretation of the main effects changes once you include interaction terms:

https://www3.nd.edu/~rwilliam/stats2/l53.pdf

As noted in the handout, for purposes of interpretation it is sometimes helpful to center continuous independent variables (but not essential).

[Niklas Max]

The key thing you need to understand is that your "main effect" in the interaction model means something different from the "main effect" in the first model (without interaction).

In a modl containing both DV and DV#M (where M is the variable you interact with DV), the coefficient of DV is not telling you the "main effect" of DV. It is telling you the effect of DV conditional on M = 0. Indeed, when you use an interaction model, you are positing that there is no such thing as the effect of DV. In an interaction model, DV has a different effect corresponding to each value of M. The DV#M coefficient tells you how much the effect of DV conditional on a given value of M differs from the effect of DV conditional on M = 0.

The simplest case is when M is just a 0/1 indicator variable. Then DV has two effects: one when M = 0 (given by the coefficient of DV) and another when M = 1, whose value would be the sum of the DV and DV#M coefficients. If M is a continuous variable, then the effect of DV conditional on the value of M is given by the coefficient of DV + M*coefficient of DV#M.

So what your results tell you is: 1. If you ignore the interaction of DV with M, you will find that the overall effect of DV on IV is "significant." 2. But when the interaction with M is recognized you find that the effect of DV is not significant when M = 0. It may nevertheless be "significant" for M = 1 (or, if M is continuous, for some other important values of M). You can use -lincom- to get those effect estimates. Or, better still, if you used factor-variable notation in your regression command, use -margins-.

In future posts, it is better to show the actual code you used than to describe your models in words. I could have given a much more concise, clear response to your question had you done that, and could have offered specific code to use for calculating the effect(s) of DM conditional on non-zero value(s) of M.

__________________________________________________ ________________________
Dear Clyde,

thanks a lot for this helpful answer!
I still include the codes and results of my model to get the answer more precise. Maybe your answer differs after that.

1. As I aim to show the effect of nationalism on foreign partner choice, with several control variables, this is the first thing I did:

fracreg logit partner nationalist ownership league ln_attendance ln_gdppercapita ln_population revenue2, vce (robust)

Result:
- Nationalism has a negative coefficient with a p-value of 0.032, so significant
- Control variables are mostly significant, but that doesn't matter at this stage.

2. However, when I now include my two (performance-related) moderators:

fracreg logit partner nationalist dom_performance int_performance1 interaction1 interaction2 ownership league ln_attendance ln_gdppercapita ln_population revenue2, vce (robust)

Result:
- Nationalism is still negative, but the p-value rises to 0.589, showing no significance once the moderators are included.

So again, am I allowed in my thesis to overall say Hypothesis 1 (nationalism) is significant or not?

[Clyde Schechter]

am I allowed in my thesis to overall say Hypothesis 1 (nationalism) is significant or not?

If you present the first model, then you can say that overall you found a "statistically significant" effect of nationalism. If you present the second model then you cannot say that, at least not based on what you have shown.

To give more specific advice for further analysis that might shed light on how best to present the results of the model with interactions, you need to show how the interaction1 and interaction2 variables were created. In particular, it is important to know whether the variables that were interacted with nationalism are continuous or dichotomous. And if they were dichotomous are they simply two levels of a single three-level variable, or are they two separate variables.