Model with interaction term

[Paul Leeds]

Dear All

I am estimating a random effect model where the main effect is not statistically significant. However, after including an interaction term by multiplying the main effect (continuous) with other independent variable (dummy, 1 for year 2008-2001 and 0 for 2005-2007 to control for the economic crisis) the interaction is significant and makes the main effect also significant.
Does that sounds right? and how can I interpret the second model (with the interaction)?
The interaction term is hypothesized based on previous literature showing their relationship.

Many Thanks

Paul

[Richard Williams]

It would help to show your results. But sure, it can happen. One way would be if the effect of a variable was positive during one time period and negative in another. But, there are other possibilities. Again, show your results and explain what these variables are and people might be able to say more.

[Clyde Schechter]

In your original model, with no interaction term, you were (implicitly) imposing the constraint that there was an effect of your continuous variable on the outcome, and that that effect was the same over both the 2005-2007 and 2008-2001 (do you mean 2008-2011?) periods. The analysis responded by saying, in effect: well, under that constraint, I can find no effect. Another way of saying that is that an average effect over both time periods is either too small or too imprecisely determined by the data to say with confidence that it is non-zero.

When you add the interaction term to the model, you are saying: maybe there isn't a single effect that applies at both time periods--perhaps there is one level of effect in 2005-2007 and a different level of effect in 2008-2011. The analysis responded by agreeing with this. In the new model output, the coefficient of the continuous main effect estimates the effect during 2005-2007 (i.e. when the dummy variable is zero.) An estimate of the effect of the continuous variable in 2008-2011 (dummy == 1) does not appear directly in the output: it is the sum of the coefficients of the continuous variable and the interaction term. You can estimate it using -lincom- (among other ways).

Since the model with no interaction had no significant main effect, it is likely that the effects during the two time periods are in opposite directions, which caused the model lacking an interaction term, to find no net effect.

[Paul Leeds]

Hello Richard,
Thanks for your contribution.

Clyde, Thanks a lot for your detailed reply.
Yes, I mean 2008-2011.

Does the positive coefficient of the interaction term indicate the direction of the effect? i.e. since it is positive it may indicate that the relationship between the interaction term and the dependent variable is significant only in 2008-2011?

Do we always need to ignore the main effect in the model with the interaction or do I need to interpret the significance of the main effect after including the interaction term?

One last question. Does estimating the model on both categories (2005-2007 and 2008-2011) by including for example (if the dummy 1 and then if 0 in STATA) to examine the effect in both directions instead of lincom?

Many Thanks

[Richard Williams]

Paul, this is one of many handouts that may help you to understand main effects and interactions. I recommend it, not because it is so good, but because it is so convenient (for me).

http://www3.nd.edu/~rwilliam/xsoc63993/l53.pdf

There are handouts before and after that one that also deal with interaction effects. If interested, see

http://www3.nd.edu/~rwilliam/xsoc63993/index.html

[Richard Williams]

You may also want to consider adding an X^2 term in addition to, or in place of, the interaction of dummy and X. Why?

1. When X^2 belongs in the model, the relationship between X and Y is curvilinear e.g. during part of the range of X increases in X produce increases in Y, but in other parts of the range of X increases in X produce decreases in Y. If you fail to include X^2, then these positive and negative effects can offset, possibly making it look like X has no effect at all on Y. Hence, like adding an interaction term, adding X^2 can suddenly make the effect of X come to life.

2. As luck would have it, I was discussing an example just today where adding X^2 caused the interaction of gender and X to become insignificant. Why? Well, it turned out that men were disproportionately likely to have values of X in the range where X increased Y, Women, on the other hand, were more likely to have values of X in the range where increases in X produced decreases in Y. For women, then, cases where X was in the positive range were somewhat offset by cases where X's values were in the negative effect range, more so than was the case for men. The net result was that, once you added an interaction for gender, the effect of X appeared to be only half as large for women as it was for men. However, once you added the X^2 term, the effect of X and X^2 was the same for men and women. (This same example is discussed briefly in section 4.5 of http://www.stata-journal.com/sjpdf.h...iclenum=st0208)

3. Finally, it may make more sense conceptually to believe that the effect of X on Y is curvilinear, than it does to believe that the effect of X on Y suddenly changed dramatically, even reversing signs, across time.

I don't know if this applies in your case. But, sometimes the apparent presence of an interaction term may actually reflect the fact that the model is misspecified in some way.

[Paul Leeds]

Hello again Richard,

Many thanks for the comments and materials. I will of course go through them carefully.

I don't know whether this helps. when I estimate the model with only 0 dummy, i.e. 2005-2007, the coefficient of X becomes significant (p-value .012 and negative coefficient) which support the hypothesis (provided that one insignificant control variable is removed from the model). However, when estimating the model with the 1 dummy (years 2008-2011) the coefficient becomes insignificant. Does that indicate that it may be a curvilinear relationship case?

By the way, what is X^2 term?

Regards

[Richard Williams]

X^2 means square the value of X. If X^2 is in the model the relationship between X and Y is U-shaped (or upside down U).You can do this via something like

reg y x c.x#c.x

I don't understand what you did. It shouldn't make any difference whether you use the 0 dummy or the one dummy -- they are perfectly negatively correlated with each other. Maybe you could post some of your code and output so it would be clearer what you are doing and what results you are getting.

To further clarify X^2 -- Assume health is measured on some kind of continuous scale. The relationship between calories and health will likely be curvilinear. If you have 0 calories you starve to death. Take in 10,000 calories a day and you can suffer from problems of obesity. So more calories are good for your health up to a certain point, and after that more calories start becoming harmful.

[Paul Leeds]

I have added the X^2 term to the model after creating the new variables as:

gen SQNAF = sqrt(NAF)

where NAF is the main independent variable.

However, in the new model (keeping the interaction term and the main effect) the coefficient of SQNAF is not significant but the main effect remains significant.

Here are the results:

. xtreg READ SQNAF NAF (CONTROL VARIABLES) CRISIS NAF*CRISIS, re vce(cluster SIC) Random-effects GLS regression Number of obs = 1319 Group variable: isinnum Number of groups = 195 R-sq: within = 0.0294 Obs per group: min = 3 between = 0.0878 avg = 6.8 overall = 0.0548 max = 7 Wald chi2(13) = 543.17 corr(u_i, X) = 0 (assumed) Prob > chi2 = 0.0000 (Std. Err. adjusted for 22 clusters in SIC) --------------------------------------------------------------------------------------------------------------- Robust READ| Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+------------------------------------------------------------------------------------------------ SQNAF | .008914 .0123975 0.72 0.472 -.0153846 .0332125 NAF | -.4735469 .1839156 -2.57 0.010 -.8340147 -.113079 CRISIS | -3.962541 1.18395 -3.35 0.001 -6.28304 -1.642043 NAF*CRISIS | .4079129 .2007749 2.03 0.042 .0144013 .8014245 _cons | -34.76328 83.77859 -0.41 0.678 -198.9663 129.4397 -------------+----------------------------------------------------------------------------------------------- sigma_u | 3.5728195 sigma_e | 5.4257627 rho | .30246084 (fraction of variance due to u_i)

READ is the quality of text where low means difficult to comprehend
NAF is the fees paid to qualified auditor to audit the text
the hypothesis is that higher levels of fees may lower the text quality

Now, does that make sense?

[ben earnhart]

squared term is not "gen SQNAF = sqrt(NAF)" but rather, "gen SQNAF=NAF^2".

[Richard Williams]

Paul, the output would be much easier to read if you used the code formatting. Click on the A that appears next to the smilie that appears on the upper right when writing a message. Then click on #. That will insert a thing for code, and you can paste your output there.

[Paul Leeds]

Sorry Richard I thought that would be it to paste the table.

Now, I have added the X^2 term to the model after creating the new variables as:

gen SQNAF = sqrt(NAF)

where NAF is the main independent variable.

However, in the new model (keeping the interaction term and the main effect) the coefficient of SQNAF is not significant but the main effect remains significant.

Here are the results:

for simplicity reasons I have omitted the other control variables.

 xtreg READ  NAF SQNAF (CONTROL VARIABLES) CRISIS  c.NAF#CRISIS, re vce(cluste
> r SIC)

Random-effects GLS regression                   Number of obs      =      1319
Group variable: isinnum                         Number of groups   =       195

R-sq:  within  = 0.0294                         Obs per group: min =         3
       between = 0.0878                                        avg =       6.8
       overall = 0.0548                                        max =         7

                                                Wald chi2(13)      =    543.17
corr(u_i, X)   = 0 (assumed)                    Prob > chi2        =    0.0000

                                   (Std. Err. adjusted for 22 clusters in SIC)
------------------------------------------------------------------------------
             |               Robust
      READ|      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         NAF |  -.4735469   .1839156    -2.57   0.010    -.8340147    -.113079
       SQNAF |    .008914   .0123975     0.72   0.472    -.0153846    .0332125
        
      CRISIS |  -3.962541    1.18395    -3.35   0.001     -6.28304   -1.642043
             |
CRISIS#c.NAF |
          1  |   .4079129   .2007749     2.03   0.042     .0144013    .8014245
             |
       _cons |  -34.76328   83.77859    -0.41   0.678    -198.9663    129.4397
-------------+----------------------------------------------------------------
     sigma_u |  3.5728195
     sigma_e |  5.4257627
         rho |  .30246084   (fraction of variance due to u_i)
------------------------------------------------------------------------------

READ is the quality of text where low means difficult to comprehend
NAF is the fees paid to qualified auditor to audit the text
CRISIS is a dummy, 1 for 2008-2011 and 0 for 2005-2007 to control for the economic crisis
the hypothesis is that higher levels of fees may lower the text quality

Now, does that make sense?

[Richard Williams]

Ok, so much for my theory that the squared term might work better than the interaction. (But keep that in mind for futurue work!)

Incidentally you don't need to compute the squared term separately. Instead do

xtreg READ  NAF c.NAF#c.NAF CRISIS  c.NAF#CRISIS, re vce(cluster SIC)

It would now help to see the results from the model that had NAF and then the model that added the interaction. But in the current case, the effect of NAF is -.47 in the first time period, and -.47 +.4079 = about -.06 in the second. So, if you got something similar in the model without the squared term, that would suggest NAF has a strong negative effect in the first period and almost no effect in the 2nd. When you fail to include the interaction, the strong effects in the first time period get diluted by the weak effects in the 2nd, possibly leading to the mistaken conclusion that NAF doesn't have an effect in either time period.

I think this thread started by noting that what you were seeing could be due to a reversal of signs across time periods. But, it could also be due to changing magnitudes of effects, as strong effects in one period get diluted by weak effects in another period.

Does that make sense, and if so does it conveniently tie in with your theoretical expectations?

Again, rerun the models without the now-distracting squared terms (or maybe that is what you started with in the first place -- we probably should have said right away to post the results) and see if you still get this phenomenon of strong effects in one time period and little or no effect in the other.

[Rich Goldstein]

actually, Paul Leeds used the square root rather than the square, as already pointed out in #10 above;

[Richard Williams]

the hypothesis is that higher levels of fees may lower the text quality

Again, we'd have to see the results without the squared term, but as it stands it appears that the hypothesis holds before the crisis but not afterwards. Before the crisis, higher fees result in lower text quality, but after the crisis, fees have little or no effect on text quality.