quadratic interaction with categorical variable

[William Salazar]

Hi Stata people,

I want to look at a squared interaction term of a variable of interest Z in my FE regression. I would like to see how this relationship varies for different plausible values of Z for different categories in my data (sectors of the economy) .
For some sectors I see different relationships between Y and Z when Z is low, medium, and of high value which is interesting for my work.

Two questions:

• Q1: Is this approach empirically sound? See a MWE using the nhanes2 data below that replicates my approach.

• Q2: When comparing different categories, I want to compare only the results that are significant on the dydx marginsplot. I see publications that use squared interaction terms but then plot the full conditional margins effects but do not look at the dydx/conditional marginal effects or their significance. This seems strange to me. There seems value in looking/interpreting only at ranges of Z that the is a significant relationship with Y (i.e. just focus on the significant margins dydx results) when interpreting results. Am I missing something here? The MWE illustrates.

MWE nhanes2:

webuse nhanes2

*regression to structured in a similar way to my own -  just not a panel FE:

reg bmi i.race bpsystol race#c.bpsystol race#c.bpsystol#c.bpsystol

Let us assume that both interaction terms are significant using testparm.
now want to compare how bpsytol's impact on bmi for different levels of bpsystol for difference races:

margins race if inlist(race,1,3),dydx(bpsystol) at(bpsystol=(100(10)150))
marginsplot, level(99)

This produces a plot (where I set CI to be 99% to induce an insignificant slope to aid my example - at bpsystol is 150):

In my work I would want to focus on a plot such as this, and would interpret it as there is a positive relationship between bpsystol and bmi which is reducing as bpystol increase for both "White" and "Other" but persists further (150 levels of bpsyol). for White but not "Other".

[Clyde Schechter]

In my work I would want to focus on a plot such as this, and would interpret it as there is a positive relationship between bpsystol and bmi which is reducing as bpystol increase for both "White" and "Other" but persists further (150 levels of bpsyol). for White but not "Other".

But that's clearly not true. Just look at the graph. It does persist for Other. In fact, the rate of decrease is the same all the way out to SBP = 150, and the difference between White and Other is continuing to grow. The fact that the marginal effect is not "statistically significant" has nothing to do with the particular statement you are making about the interaction.

I think, though perhaps I am not understanding your reasoning, you are making the very common mistake of thinking "effect not statistically significant" means "no effect." It does not. That is a widespread fallacy, largely attributable to poor quality teaching of the meaning of statistical significance. This kind of thing is one of the reasons that many statisticians, with the endorsement of the American Statistical Association leadership, have called for eliminating the use of statistical significance altogether. See https://www.tandfonline.com/doi/full...5.2019.1583913 for the "executive summary" and https://www.tandfonline.com/toc/utas20/73/sup1 for all 43 supporting articles. Or https://www.nature.com/articles/d41586-019-00857-9 for the tl;dr. But even if you still believe that statistical significance is a useful concept, you need to use it correctly and avoid this kind of mistake.

[William Salazar]

Thanks so much for the reply Clyde.
Those links were super-interesting. I haven't gone too far down the rabbit hole with them but will take a further look!

Keeping my blinkers on and sticking with statistical significance, I'm just trying to see where I've made a mistake (I'm still a newbie at interactions though I've read around - I worry now I have a fundamental misunderstanding):

First a point of clarification - when you said:

Just look at the graph. It does persist for Other. In fact, the rate of decrease is the same all the way out to SBP = 150

Unless I'm confused I think we agree - I said "Other" did persist... just not as "far" as for "White" which has significance up to 150 bpsyol while "Other" does not.

I must be missing something and wonder if the missing piece for me is in your reply where you say:

. The fact that the marginal effect is not "statistically significant" has nothing to do with the particular statement you are making about the interaction.

In my italic statement I had thought I was just referencing the marginal effect and its interpretation. Would you be able to explain what the marginal effect Other/SBP=150 not being statistically significant implies (in this context, and putting faith in statistical significance.)?

My own likely incorrect thinking: the marginal effect not being "statistically significant" for Other/bpsyol=150 would mean there is only sufficient confidence that there a positive relationship between bpsyol and bmi for Other when bpsyol<150.

[Clyde Schechter]

My own likely incorrect thinking: the marginal effect not being "statistically significant" for Other/bpsyol=150 would mean there is only sufficient confidence that there a positive relationship between bpsyol and bmi for Other when bpsyol<150.

A more correct interpretation would be "At systolic BP = 150, the data and model are inconclusive as to whether the marginal effect of systolic BP on BMI among the Other group is positive, negative or zero."

But that, too, has nothing to do with your original statement about the trend where marginal effect of systolic BP decreases as systolic BP increases. That is clearly true, regardless of whether the marginal effect at systolic BP = 150 among the other group is positive, negative, or zero: regardless, it is clearly less than it was at systolic BP = 140, and it is also clearly less than that in the White group. Those things remain true even though the direction of the marginal effect is indeterminate.

[William Salazar]

Thanks again for the reply Clyde.

In my actual context I want to look at different marginal effects plots of a firm's financial performance as a function of a certain social measures. The specification is similar to the above with squared interaction terms where:
race = sector (e.g. energy, services..) (sec)
bmi = firm financial performance (Perf)
bpsystol = social measure (S).

reg Perf i.sec S sec#c.S sec#c.S#c.S

I wanted to be able to use the marginal effect plots to say for what ranges of (S) there was a positive, negative, or no association with firm financial performance. We might expect to see sectors present differently given the characteristics of the sector.
Some might only be related to S when it is high, some might be negatively related to S (as it costs to do well in S, and if customers don't care about S in the sector a high S would worsen firm performance), and so on.


For example, in plot below for one sector only, my possibly erroneous thinking is that there is only evidence of a positive association when S is 60 and above for the services sector - so for this sector I would say there is evidence of a positive link between S and firm performance when S>=60 and thus with our level of confidence we find that it takes an S of 60 or above to see a benefit in firm performance. I was thinking just to report significant results as per yellow shading in sector plots given the amount of results/sectors I have:

I'm not 100% confident on this, and might need to hit the books to find out more!

[Clyde Schechter]

I would say there is evidence of a positive link between S and firm performance when S>=60 and thus with our level of confidence we find that it takes an S of 60 or above to see a benefit in firm performance.

What I would say is that there is evidence of a positive link between S and firm performance when S>= 60, and that for S < 60 there may or may not be a positive association.

[William Salazar]

Excellent. Again thanks for all the feedback!