U-test Outcomes interpretation

[Zainab Mariam]

Question 1:

. xtreg Y X1 X2 X3 X4, fe

Y	B	P
X1	.0003274	0.000
X2	.0276111	0.000
X3	-.0466578	0.000
X4	.0316747	0.000
_cons	-.1327554	0.000

. xtreg Y X1 X1Squrd X2 X3 X4, fe

Y	B	P
X1	.0001697	0.016
X1Squrd	7.82e-07	0.026
X2	.0298372	0.000
X3	-.0611631	0.000
X4	.0318192	0.000
_cons	-.1604234	0.000

utest X1 X1Squrd, fieller
(48,183 missing values generated)
(48,183 missing values generated)

Specification: f(x)=x^2
Extreme point: -108.4631
Test:
H1: U shape
vs. H0: Monotone or Inverse U shape

	Lower bound	Upper bound
Interval	.0000149	241.0008
Slope	.0001697	.0005467

Extremum outside interval - trivial failure to reject H0
95% Fieller interval for extreme point: [-1602.7879; -11.067647]

Do these findings mean that there is Monotone relation between X1 and Y? Or does this mean there is Inverse U Shape relation between X1 and Y?


Question 2:

. xtreg Y X1 X2 X3 X4, fe

Y	B	P
X1	.9206401	0.000
X2	.0644442	0.000
X3	-.0452923	0.000
X4	.0210274	0.011
_cons	-.3993418	0.000

. xtreg Y X1 X1Squrd X2 X3 X4, fe

Y	B	P
X1	1.690577	0.000
X1Squrd	-12.30493	0.000
X2	.0639376	0.000
X3	-.0454351	0.000
X4	.0214638	0.009
_cons	-.4015362	0.000

utest X1 X1Squrd, fieller
(47,635 missing values generated)
(47,635 missing values generated)

Specification: f(x)=x^2
Extreme point: .0686951
Test:
H1: Inverse U shape
vs. H0: Monotone or U shape

	Lower bound	Upper bound
Interval	.000525	.080509
Slope	1.677657	-.2907374
t-value	7.655816	-.8889454
P>|t|	9.94e-15	.1870204

Overall test of presence of a Inverse U shape:
t-value = 0.89
P>|t| = .187
95% Fieller interval for extreme point: [.05450081; .10825843]

I kindly ask you please how to interpret these results. What is the relation between X1 and Y?

Question 3:

X1 was significant before adding X1Squrd to the regression model, while X1 became insignificant when adding X1Squrd to the model. In this case, is it right to continue checking for non-linearity between X1 and Y?

[Clyde Schechter]

For Question 1, Stata says

Test:
H1: U shape
vs. H0: Monotone or Inverse U shape
-------------------------------------------------
| Lower bound Upper bound
-----------------+-------------------------------
Interval | .0000149 241.0008
Slope | .0001697 .0005467
-------------------------------------------------

Extremum outside interval - trivial failure to reject H0

(emphasis added)

So you do not reject the hypothesis of monotone or inverse U-shape. In accordance with general principles of hypothesis testing, in the absence of a prior power calculation, you cannot say whether the failure to reject H0 is due to the absence of a U-shape or do to the data being too noisy or sparse to detect it.

For Question 2:

Test:
H1: Inverse U shape
vs. H0: Monotone or U shape
-------------------------------------------------
| Lower bound Upper bound
-----------------+-------------------------------
Interval | .000525 .080509
Slope | 1.677657 -.2907374
t-value | 7.655816 -.8889454
P>|t| | 9.94e-15 .1870204
-------------------------------------------------
Overall test of presence of a Inverse U shape:
t-value = 0.89
P>|t| = .187

So with a p-value of 0.187, you would not reject the null hypothesis of monotone or U-shape at conventional significance levels. Again, whether this is due to absence of an inverse U-shape relationship or inability of the data to detect one is indeterminate.

Question 3. Once you introduce a quadratic term into the model, significance tests for the linear term by itself are completely meaningless and you should not even look at that result, let alone waste time trying to interpret it. There is no effect of X separate from that of XSquared, nor vice versa. The only meaningful test here (if you believe any significance tests are meaningful at all--a topic for another day) is the joint test of both X and XSquared:

test X XSquared

[Zainab Mariam]

Dear Professor Clyde Schechter,
Thank you for your reply.

A) I did not get what you mean by “a prior power calculation”.

B) In question 1, Stata outcomes show “Extremum outside interval - trivial failure to reject H₀” but Not report P-value, while Stata outcomes report P-value in question 2. Thus, what is the difference between the two Stata outcomes? i.e., why does not Stata report the P-value in the first outcomes?

C) I applied test X1 X1Squared and got the following outcomes:
(1) X1 = 0
(2) X1Squared = 0
F(2, 25812) = 48.94
Prob > F = 0.0000
What should I conclude?

Thank you in advance.

[Clyde Schechter]

A) Power analysis calculates the probability of rejecting the null hypothesis when the alternate hypothesis is true. If the power is high, then when an effect (in this case, a U-shaped relationship) actually is present, you have a high probability of finding statistically significant results. When power is low, the effect is likely to be missed. Power depends on the sample size as well as the variation within the data. Stata includes the -power- command which can do this kind of calculation for some models. I don't know if it can do it for this particular kind of problem. Other software that specializes in statistical power calculations may be needed for this.

B) It does not report a p-value here because when it fits the quadratic model, it finds that the value of X1 at the turning point lies outside the range of the data. So that is enough to conclude that within the data the relationship is not U-shaped. No p-value is required for this.

C) It means that your data are not compatible with a model in which Y is independent of X, and are more compatible with a model in which there is a linear or quadratic relationship between Y and X.

[Zainab Mariam]

Dear Professor Clyde Schechter,
Thank you for your clarification.