Drawing a graph with insignificant coefficients!

[Eldar Saleh]

Hi,

Can we draw a graph when some of our regression coefficients show up as insignificant?
Specifically, I have estimated a quadratic moderation relationship in which only the interaction effect and the coefficient of the independent variables were significant. Then I wanted to plot a curve using the Johnson-Neyman toolbox

Consider for example these two equations:
Eq 1. Y = a + b1*X + b2*X^2 + e
Eq 2. Y = a + b1*X + b2*X^2 + b3*W + b4*X*W + b5*X^2*W + e
where, Y is DV (Dependent Variable), X is IV (Independent Variable), and W is the moderator.

I really doubt whether I can plot these equations when b1 is insignificant in Eq1, and b2, b3, and b4 are insignificant in Eq2.
I understand that the above questions include separate issues. First, I don't want to use the result of this discussion for the matter of inference. It is just for representing a graph to facilitate communication. Second, Equations 1 and 2 are different, apparently. However, the issue is if we are going to insert data into variable X, can are we allowed to do it for terms whose coefficients are insignificant?

Thank you in advance.

[Bruce Weaver]

Hello Eldar Saleh. Welcome to the forum.

First, this appears to be the same question you posted earlier. To keep things tidy, I suggest you go and delete that other thread. Thanks.

I think you are asking if you can plot the fitted values from a model in which one or more coefficients are not statistically significant. If that is the question, then yes, you can plot the fitted values. From what you've said, I wonder if you want to compare the fitted value plots from both models. Apologies if I have misunderstood your question.


PS- I see no mention of Stata in that article J-N toolbox article you linked to. But you can likely achieve what you want using -margins- and -marginsplot-. See the simple example in section 3.3 of the review of Michael Mitchell's IVRMUS that Angela MacIsaac and I published in the Stata Journal.

[Eldar Saleh]

Dear Bruce,

Thank you for your reply.

I just edited my post title but not posted twice, so I don't see more than one post! there may be a bug or I am new to the forum and not familiar with some details.

Let me rephrase and simplify my question. Suppose that in the simple equation 1, if the coefficient b1 is insignificant, the curve is not representing full curvilinearity for all data of X, then if I plot a curve based on the insignificant coefficients I implicitly assume that I have the full curvilinearity across all data of X. In fact, the question is whether we can draw a curve that its optimum occurs on the vertical axe (because the coefficient b1 is insignificant) or we should draw a curve supposing that b1 is significant and we can multiply it by the values of X.

[Eldar Saleh]

Thanks a lot to Clyde Schechter
He responded to my question in the other similar post (mistakenly posted twice). I past his answer here and appreciate it if he joins us to continue this discussion here from now on:

Clyde Schechter said:

" In a quadratic model, the coefficient of the linear term says nothing whatsoever about the "significance" of the variable X in the model. In fact, all it tells you is whether the location of the parabola's axis of symmetry is located nearer to or farther from the y-axis! It is a very rare research question indeed that would lead one to care about that. When dealing with a quadratic model, you should never consider the linear term in isolation. If you must worry about the "significance" of the X variable, that has to be based on a joint test of the linear and quadratic terms--never on either alone."

The research question that I am interested in is whether challenge stressors have an inverted U shape relationship with job performance. Therefore, the level of challenges that leads to a return is critical for my research. When the linear term has no significant coefficient, it is not possible to support the hypothesis that when challenges increase from a lower level to a medium level, performance will increase as well. If I want to represent this in a graph, I was not sure whether I can use the insignificant linear term to draw or not.

[Clyde Schechter]

The significance or lack thereof of the linear coefficient is totally irrelevant to whether you have an inverted U-shape in your data. To establish a quadratic model, you need to do a joint test of the linear and quadratic terms. Then, to establish that you have an actual U or inverted-U, as opposed to just a curvilinear shape, you need to see if the turning point of the quadratic lies inside the range of your X variable. You can do that with

nlcom -_b[X]/(2*_b[X^2]) // REPLACE X and X^2 BY ACTUAL VARIABLE NAMES USED

If that falls squarely within the range of observed values of X, you have a U (if coeff of X^2 is positive) or inverted-U (if coeff of X^2 is negative). That is how you do that within the framework you are working in.

That framework itself is somewhat questionable. For example, the function y = log(x), if fitted with a quadratic model over a range of, say, 1 to 100 will pass this test for being an inverted-U relationship, which, clearly, it is not. A more stringent test is to establish that the coefficient of Y vs X is positive at the low end of the range of X, and negative at the high end of the range of X. Difficulties in applying this arise, however, because it is complicated to define the low and high ends of the range, so most people don't use this approach, even though it is actually more correct.

Finally, I will just add that you appear to have been taught that "not statistically sigificant" = "no effect." This is an egregiously wrong interpretation of statistical significance. It is widely taught, and this gross misunderstanding is so widespread that it is one of the principal reasons that the leadership of the American Statistical Association has recommended that the use of statistical significance itself be abandoned. See https://www.tandfonline.com/doi/full...5.2019.1583913 for the "executive summary" and https://www.tandfonline.com/toc/utas20/73/sup1 for all 43 supporting articles. Or https://www.nature.com/articles/d41586-019-00857-9 for the tl;dr. You should make every effort to disgorge that fallacy from your brain.