When should quadratic terms be retained if diagnostics support nonlinearity but theory and fit gains are weak?

[Cassie Ding]

Dear Statalist members,
I am estimating panel-data models using PPML with high-dimensional fixed effects. I consider three continuous explanatory variables , and estimate the model separately for several broad subsamples (“groups”).
For each group × variable combination, I compare a quadratic specification against a restricted model that excludes the squared term but retains the linear term.
Importantly, there is no strong or widely accepted theoretical consensus in the literature that these explanatory variables should follow an inverted-U (or U-shaped) relationship.Among the three variables, x1 (analogous to a location-quotient–type measure) may have some intuitive scope for diminishing returns, although even for this variable the existing applied literature rarely includes quadratic specifications explicitly.My initial modeling consideration therefore focused primarily on this variable. However, because all three variables are core explanatory variables in the analysis, I also explored quadratic terms for the remaining two variables, partly for symmetry and to avoid concerns that nonlinearities were selectively introduced only for a subset of regressors.At the same time, this raises a substantive methodological question: in the absence of clear theoretical or empirical precedent, is it preferable to allow nonlinear terms only for those variables where they are conceptually motivated, or to treat all key explanatory variables symmetrically even if the nonlinear specification is largely exploratory?
Reported diagnostics：For each group–variable pair, I report:1.Linear and squared coefficients (with significance);2.The implied turning point and whether it lies within the sample support;3.Marginal effects evaluated at the sample minimum and maximum with significance tests;4.Changes in model fit relative to the restricted (linear-only) model:Log pseudolikelihood gain (absolute and percentage),BIC reduction,Change in correlation between observed and predicted values.
Below is an excerpt of the results :

Group	Var	b_linear	b_squared	TurningPt	TP_Loc	Slope@Min	Slope@Max	LL_Gain	LL_%	BIC_Red	Corr_Gain	Quad_OK
group1	x1	0.216***	-0.007**	15.47	Out(>Max)	0.225***	0.021	107.73	0.10%	205.3	-0.00062	No: TP outside
group1	x2	1.181***	-1.770***	0.81	InRange	2.865***	-2.004***	1792.56	1.63%	3575	0.00568	Yes: inv-U
group1	x3	1.054***	-0.420**	2.51	Out(>Max)	2.107***	0.540**	48.71	0.05%	87.3	0.00015	No: TP outside
group2	x1	0.411***	-0.018***	12.12	HighTail	0.442***	-0.069***	1096.18	2.46%	2182.1	0.00762	Borderline
group2	x2	-0.509**	0.876***	1.01	InRange	-1.770***	0.946***	215.89	0.50%	421.5	0.00196	Yes: U-shape
group2	x3	0.580***	-0.27	2.33	Out(>Max)	1.257*	0.249	4.18	0.01%	-1.9	0.00004	No: no fit gain
group3	x1	0.338***	-0.007	26.21	Out(>Max)	0.352***	0.164**	8.54	0.05%	7.9	-0.00031	No: sq insignificant
group3	x2	0.617**	-0.425**	2.59	InRange	2.200**	-0.216	10.02	0.06%	10.9	0.00004	Borderline
group3	x3	-0.265*	0.490***	1.52	InRange	-1.493***	0.335	13.09	0.08%	17	0.00023	Borderline
group4	x1	0.384***	-0.024***	9.22	HighTail	0.443***	-0.231***	1428.98	1.37%	2846.8	0.00633	Yes: strong inv-U
group4	x2	0.831**	-0.710**	2.45	InRange	3.477**	-0.562	58.49	0.06%	105.8	-0.00023	Borderline
group4	x3	0.279	-0.364	1.64	InRange	1.191*	-0.167	19.73	0.02%	28.3	0.00048	No: insignificant

My questions：
My own view is that the following joint diagnostic rule should be applied mechanically:If the turning point lies outside the sample support, or if marginal effects at the sample boundaries do not show a meaningful and statistically significant sign change, the quadratic term should be excluded.My uncertainty concerns the remaining cases.
Q1.
When joint diagnostics support nonlinearity (turning point in range; slope@min and slope@max differ in sign and are significant), but
(i) theoretical support is weak or ambiguous, and
(ii) improvements in fit (LL gain, BIC reduction, correlation gain) are quantitatively small,
is it still considered good practice to retain the quadratic term?Relatedly, are there rules of thumb, benchmarks, or references discussing how large LL or information-criterion improvements should be before a nonlinear term is considered substantively meaningful in applied work?
Q2.
For the same variable, some groups clearly satisfy the joint nonlinearity diagnostics while others do not.Is it acceptable to include quadratic terms only for the groups where diagnostics support them, or should one impose a uniform functional form across groups for comparability, even when the quadratic term is evidently inappropriate in some subsamples?
Q3.
Are there recommended practices or references for presenting and justifying such mixed evidence on nonlinearity when nonlinear effects are not strongly theory-driven?

Appendix: key Stata implementation details
Below are simplified excerpts of the Stata logic used to implement the diagnostics:

* Center and square
gen c_x  = x - mean_x
gen sq_x = c_x^2

* Turning point
scalar tp = mean_x - _b[c_x] / (2 * _b[sq_x])

* Marginal effects at sample bounds
lincom _b[c_x] + 2 * _b[sq_x] * (min_x - mean_x)
lincom _b[c_x] + 2 * _b[sq_x] * (max_x - mean_x)

* Restricted model (drop squared term only)
ppmlhdfe y c_x controls, absorb(id year) vce(cluster id)

I include this only to verify that the diagnostic logic itself is sound.
Thank you very much for any guidance or references.

[Jianli Ding]

Regarding Q1, I do not endorse the rule that a nonlinear term must be retained solely because it is statistically significant. The decision to retain a quadratic term should be driven by the interpretative goals of the model rather than statistical tests alone.As for Q2, I would recommend reporting the linear specification for all groups. You can then separately discuss those specific groups that exhibit strong nonlinearity.I did not fully understand your third question.

[Cassie Ding]

Regarding Q1, I do not endorse the rule that a nonlinear term must be retained solely because it is statistically significant. The decision to retain a quadratic term should be driven by the interpretative goals of the model rather than statistical tests alone.As for Q2, I would recommend reporting the linear specification for all groups. You can then separately discuss those specific groups that exhibit strong nonlinearity.I did not fully understand your third question.

Thank you very much for your insightful comments.

Regarding the issue of model specification (and to clarify my previous concerns), I have a follow-up question:If statistical tests (e.g., significance of the squared term, valid turning point) suggest nonlinearity for multiple variables, but I choose to include the squared term for only one variable (perhaps due to stronger theoretical intuition) while excluding it for others (where theory is weak or the improvement in model fit is negligible), would this "selective" approach be considered biased or prone to criticism of "cherry-picking"? I am trying to find the right balance between statistical evidence and model parsimony, but I am worried about consistency within the same regression equation.

To clarify my third question: I am looking for methodological literature or classic references that provide guidelines or "decision rules" for adopting quadratic specifications when theory is silent. Specifically, are there citations that discuss how to weigh statistical significance against substantive significance (e.g., the magnitude of improvement in Log-Likelihood or BIC) when deciding whether to retain a nonlinear term?

Thank you again for your time and guidance.

[Rich Goldstein]

a couple of comments: (1) while I generally agree with Jianli Ding (#2 above), if the relationship between the 2 variables is not monotonic, then excluding some form of non-linearity may bias the results (if the relationship is monotonic, then there may not be a bias regardless of the p-value); (2) anytime you base further steps on the results of prior steps, someone will accuse you of cherry-picking or bias; in some cases prior specification of your plans may be used to counter this

[Cassie Ding]

a couple of comments: (1) while I generally agree with Jianli Ding (#2 above), if the relationship between the 2 variables is not monotonic, then excluding some form of non-linearity may bias the results (if the relationship is monotonic, then there may not be a bias regardless of the p-value); (2) anytime you base further steps on the results of prior steps, someone will accuse you of cherry-picking or bias; in some cases prior specification of your plans may be used to counter this

Hi Rich,
Apologies for the delay, it's been a busy few weeks. Thanks a lot for the help! It really helped me rethink how to justify and clarify my model, especially the part about cherry-picking.

[Cassie Ding]

a couple of comments: (1) while I generally agree with Jianli Ding (#2 above), if the relationship between the 2 variables is not monotonic, then excluding some form of non-linearity may bias the results (if the relationship is monotonic, then there may not be a bias regardless of the p-value); (2) anytime you base further steps on the results of prior steps, someone will accuse you of cherry-picking or bias; in some cases prior specification of your plans may be used to counter this

Hi Rich,
Apologies for the delay, it's been a busy few weeks. Thanks a lot for the help! It really helped me rethink how to justify and clarify my model, especially the part about cherry-picking.

[Carlo Lazzaro]

Cassie:
as a sidelight to previous helpful advice, your Q3 provides you with the answer: if the literature in your research field cannot justify your results, you should try your best to explain them yourself.