Why does an insignificant coefficient of poisson regression can make a significant non-linear graph?

[David Lu]

Hi all,

I have a cross-sectional dataset (sample size:734) and use a poisson model. The square term of iv1 is insignificant, but when I graph the interaction effect, I can observe a inverted U-shape curve. How comes it make that?

poisson sale c.iv1##c.iv2 c.iv1#c.iv1 lnctrl1 lnctrl2 i.indcode, vce(robust)
marginscontplot iv1 iv2, at1(0(1)18) at2(0(0.2)1)

Then, I add another control variable lnctrl3 into the model, the square term now is significant but the graph seems difficult to interpret. Actually, if we only observe the graphs, both of them are typical inverted U-shaped curve, but why the former yield an insignificant coefficient while the latter a significant?

poisson sale c.iv1##c.iv2 c.iv1#c.iv1 lnctrl1 lnctrl2 lnctrl3 i.private i.indcode, vce(robust)
marginscontplot iv1 iv2, at1(0(1)18) at2(0(0.2)1)

Thanks in advance,
David

[Clyde Schechter]

You have fit a quadratic model, and then plotted the resulting fit. The fact that the coefficient of the quadratic term is not statistically significant doesn't mean that it disappears when you run -margins- and -marginsplot-. The plot correctly shows the fitted model results.

The model you selected is fairly complicated, involving both an interaction between iv1 and iv2 and a quadratic term in iv2. What your model says is that you expect your outcome (after adjustment for other covariates) to exhibit a U- (or inverse-U-) shaped relationship with iv1, and that the location of the nadir (or peak) of that relationship changes depending on the value of iv2. The coefficient of c.iv2#c.iv2 carries relatively little information in this model: it tells you your parabolas are upside-down, and are relatively wide and flat. But because of the c.iv1#c.iv2 interaction term, as iv2 increases, the peak of your parabola moves from the upper end of the values of iv1 to the center of the data. That's why the iv2 = 0 curve is nearly straight, with just a little bending at the right end, whereas with iv2 = 0.8 or 1 you get a clear-cut bending of the curve right near the middle of the data.

When fitting quadratic models, the statistical significance of either linear or quadratic coefficient by itself is of limited value. If there were no iv1#iv2 interaction term, a non-significant quadratic coefficient might be an adequate reason for revising the model omitting it (though, in general, I wouldn't do that unless the peak/nadir of the curve were clearly outside the range of the observed data). But with an interaction term such as your model has, the p-value of the quadratic term by itself is of no importance and is best disregarded. The joint significance of iv1 iv2 iv2#iv2 and iv1#iv2 is what matters here.

As for the coefficient turning from not statistically significant to statistically significant when you add another variable to the model, there is nothing surprising or unusual about that. It's a different model, and everything can change.

[David Lu]

You have fit a quadratic model, and then plotted the resulting fit. The fact that the coefficient of the quadratic term is not statistically significant doesn't mean that it disappears when you run -margins- and -marginsplot-. The plot correctly shows the fitted model results.

The model you selected is fairly complicated, involving both an interaction between iv1 and iv2 and a quadratic term in iv2. What your model says is that you expect your outcome (after adjustment for other covariates) to exhibit a U- (or inverse-U-) shaped relationship with iv1, and that the location of the nadir (or peak) of that relationship changes depending on the value of iv2. The coefficient of c.iv2#c.iv2 carries relatively little information in this model: it tells you your parabolas are upside-down, and are relatively wide and flat. But because of the c.iv1#c.iv2 interaction term, as iv2 increases, the peak of your parabola moves from the upper end of the values of iv1 to the center of the data. That's why the iv2 = 0 curve is nearly straight, with just a little bending at the right end, whereas with iv2 = 0.8 or 1 you get a clear-cut bending of the curve right near the middle of the data.

When fitting quadratic models, the statistical significance of either linear or quadratic coefficient by itself is of limited value. If there were no iv1#iv2 interaction term, a non-significant quadratic coefficient might be an adequate reason for revising the model omitting it (though, in general, I wouldn't do that unless the peak/nadir of the curve were clearly outside the range of the observed data). But with an interaction term such as your model has, the p-value of the quadratic term by itself is of no importance and is best disregarded. The joint significance of iv1 iv2 iv2#iv2 and iv1#iv2 is what matters here.

As for the coefficient turning from not statistically significant to statistically significant when you add another variable to the model, there is nothing surprising or unusual about that. It's a different model, and everything can change.

Hi Clyde,

Thank you for your helpful interpretation and analysis. Yes, you're right. One minor mistake is the quadratic term is for iv1 not for iv2. I don't add an quadratic term in iv2. Moreover,as for the coefficient turning from not statistically significant to statistically significant when I add another variable to the model,what I want to know is which part of the model can account for the movement of the peak of that relationship, say, from left to right as iv2 increases?

Also, when it comes to another model, it's more complicated. It not only move from right to left horizontally but also from fall to increase vertically. As you can see from the two model, there are only iv1,iv2, c.iv1#c.iv1,and c.iv1#c.iv2 along with ctrl variables. All of them have the same sign. So, I am confused, which part of the model may account for the different movement of the peak,say, from decrease first then to increase again (a complicated horizontal and vertical movement) ?

Thanks,
David

[Clyde Schechter]

So, in general, if you have a quadratic function y = ax² + bx + c, the peak or nadir occurs at x = -b/2a. In your model b = _b[iv1] + _b[iv1#iv2]*iv2, and a = _b[iv1#iv1]. So it is the iv1#iv2 interacton term, and its negative coefficient, that drives the left-to-right (not right-to-left: look at your graphs) movement of the peak as iv2 increases.

As for how high the peak is, in the general quadratic function, the peak value is c - b²/4a. The value of c in your model is a combination of everything except the iv1, iv2#iv1, and iv1#iv1 terms. It does depend in part on iv2 through the iv2 term. But it also depends on b and a, and therefore on iv1, iv2#iv1, and iv1#iv1 terms in a somewhat complicated way. But it is easy enough to see that as iv2 increases, c increases because _b[iv2] is positive, b decreases because _b[iv1#iv2] is negative, and a doesn't change, With c increasing and b decreasing, it is not obvious what happens, because the decrease in b translates to an increase of b²/4a (because a is negative). So c - b²/4a could go either way in principle, depending on which changes more, c or b²/4a. In this case, apparently the latter dominates in the first model. In the second model, apparently the b²/4a increase dominates at lower values of iv2, but not at higher values of iv2. It is, indeed, a complicated pattern.

[David Lu]

So, in general, if you have a quadratic function y = ax² + bx + c, the peak or nadir occurs at x = -b/2a. In your model b = _b[iv1] + _b[iv1#iv2]*iv2, and a = _b[iv1#iv1]. So it is the iv1#iv2 interacton term, and its negative coefficient, that drives the left-to-right (not right-to-left: look at your graphs) movement of the peak as iv2 increases.

As for how high the peak is, in the general quadratic function, the peak value is c - b²/4a. The value of c in your model is a combination of everything except the iv1, iv2#iv1, and iv1#iv1 terms. It does depend in part on iv2 through the iv2 term. But it also depends on b and a, and therefore on iv1, iv2#iv1, and iv1#iv1 terms in a somewhat complicated way. But it is easy enough to see that as iv2 increases, c increases because _b[iv2] is positive, b decreases because _b[iv1#iv2] is negative, and a doesn't change, With c increasing and b decreasing, it is not obvious what happens, because the decrease in b translates to an increase of b²/4a (because a is negative). So c - b²/4a could go either way in principle, depending on which changes more, c or b²/4a. In this case, apparently the latter dominates in the first model. In the second model, apparently the b²/4a increase dominates at lower values of iv2, but not at higher values of iv2. It is, indeed, a complicated pattern.

Hi Clyde,

Yes, you're right. So, in the first model, instead of using the concept of peak, can I interpret the results in this way: there is a threshold exist (approximately iv1=5.8),when the value of iv1 is lower than 5.8, as iv2 increases, a firm sales (dependent variable) goes up but when iv1 goes beyond 5.8, the increase in the value of iv2 will drop down the sales?

Thanks,
David

[Clyde Schechter]

can I interpret the results in this way: there is a threshold exist (approximately iv1=5.8),when the value of iv1 is lower than 5.8, as iv2 increases, a firm sales (dependent variable) goes up but when iv1 goes beyond 5.8, the increase in the value of iv2 will drop down the sales?

Not quite. That's an oversimplification. What is true is that for any given values of iv2, there is a threshold value of iv1 such that when iv1 is below that threshold, increases in iv1 correspond to increases in firm sales, but past that threshold increasing iv1 is associated with decreasing firm sales. The value of the iv1 threshold depends on the value of iv2. As iv2 increases from zero to 1, the iv1 threshold value moves to the left, and the corresponding attained peak in firm sales moves down.

I should also note that my presentation in #4 is somewhat incomplete. In describing the second model I was assuming that b is always >= 0. Things work differently if b < 0. So the complications are even greater.

Finally, I should point out that because the Poisson model uses a log link, the curvature in the relationships at the fixed-effects predictor level gets magnified at the level of the actual outcome.

[David Lu]

Hi Clyde,

Yes, you're right. So, in the first model, instead of using the concept of peak, can I interpret the results in this way: there is a threshold exist (approximately iv1=5.8),when the value of iv1 is lower than 5.8, as iv2 increases, a firm sales (dependent variable) goes up but when iv1 goes beyond 5.8, the increase in the value of iv2 will drop down the sales?

Thanks,
David

Yes, it makes sense and if that is the case, then this result imply that it'd better keep iv2 as low as possible because the maximum value of y are always lower than that in the case when iv2=0. However, despite the threshold, we can also see that the intersection also has its implication. How can we interpret the intersection point among these curve instead of the peaks of each?

Thanks,
David

[Clyde Schechter]

I don't see the intersection point as having any particular importance here. Basically, you can think of your outcome as being a function of iv1 and iv2 (with some effects of other variables that we will ignore here). It is quadratic in iv1, with the details of the quadratic depending on iv2, call it f(iv1, iv2). Now, the intersection point has an iv1 value where the outcome is the same regardless of the value of iv2. So that is equivalent to saying that the partial derivative of f with respect to iv2 is 0, for all values of iv2 when iv1 equals that value. I won't write out all the algebra here because it's too tedious to do in this platform, but I've done it on paper. The solution is iv1 = -_b[iv2]/_b[iv1#iv2]. For that value of iv1, the outcome is the same regardless of the value of iv2. You can in fact see that for both models, this formula does in fact give the iv1-axis coordinate of the intersection point.

It's a nice exercise if you like doing the math, but I don't see any way to attach any particular importance to it. Now, if I worked in your area and knew what iv1 and iv2 are, perhaps I would feel differently about that--outcome being independent of iv2 might be important, depending on what iv2 is. But purely mathematically, I don't see it as being of interest.

[David Lu]

I don't see the intersection point as having any particular importance here. Basically, you can think of your outcome as being a function of iv1 and iv2 (with some effects of other variables that we will ignore here). It is quadratic in iv1, with the details of the quadratic depending on iv2, call it f(iv1, iv2). Now, the intersection point has an iv1 value where the outcome is the same regardless of the value of iv2. So that is equivalent to saying that the partial derivative of f with respect to iv2 is 0, for all values of iv2 when iv1 equals that value. I won't write out all the algebra here because it's too tedious to do in this platform, but I've done it on paper. The solution is iv1 = -_b[iv2]/_b[iv1#iv2]. For that value of iv1, the outcome is the same regardless of the value of iv2. You can in fact see that for both models, this formula does in fact give the iv1-axis coordinate of the intersection point.

It's a nice exercise if you like doing the math, but I don't see any way to attach any particular importance to it. Now, if I worked in your area and knew what iv1 and iv2 are, perhaps I would feel differently about that--outcome being independent of iv2 might be important, depending on what iv2 is. But purely mathematically, I don't see it as being of interest.

Hi Clyde,

Thank you for the further explanations. Yes, the intersection point itself may not be of any particular importance. However, when see the graph of the first model, it is easy to observe that the outcome (y variable) is contingent on the value of iv2 in the opposite way. On the left side, that is, if iv1 is lower than the intersection point, then iv2=1 is the best way to yield a high sales in which the sale is higher as the iv2 increases. On the contrary, on the right side, that is, if iv1 is higher than the intersection point, then iv2=1 is the worst way to yield a high sales in which the sale is lower as the iv2 increases. So, together, the intersection point shows us high-iv2 actually suits for those with low-iv1.

For instance, assume y is the amount of white blood cell, iv1 is the doze of drug A while iv2 represents the level of exposure to x-ray. The intersection point here may tell us x-ray therapy may only work better with drug A in a low-doze in increseasing the white blood cell. High doze of drug A should be taken without X-ray therapy. Does it make sense to you?

Thanks,
David

[Clyde Schechter]

Yes, I see what you're saying. That does make sense indeed!