Manual RESET test, constraints dropped.

[Simon Bensnes]

Dear all,

I am having trouble understanding why Stata drops constraints when I am conducting a test. I have searched the forum without finding any answers, so I post here. My code is appended below. I was unable to reproduce the error in the preloaded data sets, so I am using one of my own.

The problem is that I am trying to show how to perform a manual RESET test and comparing the result to the ovtest in Stata. The ovtest works fine, but when I use the manual test Stata drops to of the constraints. The test result is still reported but it is only a test of the restriction that the forth polynomial is 0, rather than testing if all polynomials are 0. My theory so far is that it has to do with the size of the coefficients somehow. The coefficients on the polynomials range from -.00009 to -3.02e-15.

Any ideas would be greatly appreciated.


reg faminc he we kl6
estat ovtest

reg faminc he we kl6
predict faminc_hat
gen faminc_hat2 = faminc_hat^2
gen faminc_hat3 = faminc_hat^3
gen faminc_hat4 = faminc_hat^4

reg faminc he we kl6 faminc_hat2 faminc_hat3 faminc_hat4
test faminc_hat2 faminc_hat3 faminc_hat4

[Richard Williams]

Simon, somebody may immediately know the answer to your question. If not, it might help if you showed both your code and output using code tags (see pt. 12 of the FAQ). You might also provide some descriptive statistics for the 4 variables in your model.

[Simon Bensnes]

Richard,

Thank you for your answer. Please see below.

Variable | Obs Mean Std. Dev. Min Max

faminc | 428 91213 44117.35 9072 344146.3
he | 428 12.61215 3.035163 4 17
we | 428 12.65888 2.285376 5 17
kl6 | 428 .1401869 .3919231 0 2



Source | SS df MS Number of obs = 428
F( 3, 424) = 30.43
Model | 1.4725e+11 3 4.9082e+10 Prob > F = 0.0000
Residual | 6.8384e+11 424 1.6128e+09 R-squared = 0.1772
Adj R-squared = 0.1714
Total | 8.3109e+11 427 1.9463e+09 Root MSE = 40160


faminc | Coef. Std. Err. t P>|t| [95% Conf. Interval]

he | 3211.526 796.7026 4.03 0.000 1645.547 4777.504
we | 4776.907 1061.164 4.50 0.000 2691.111 6862.704
kl6 | -14310.92 5003.928 -2.86 0.004 -24146.52 -4475.326
_cons | -7755.33 11162.93 -0.69 0.488 -29696.91 14186.25


. estat ovtest

Ramsey RESET test using powers of the fitted values of faminc
Ho: model has no omitted variables
F(3, 421) = 2.15
Prob > F = 0.0931


Source | SS df MS Number of obs = 428
F( 6, 421) = 16.42
Model | 1.5758e+11 6 2.6263e+10 Prob > F = 0.0000
Residual | 6.7351e+11 421 1.5998e+09 R-squared = 0.1896
Adj R-squared = 0.1781
Total | 8.3109e+11 427 1.9463e+09 Root MSE = 39997


faminc | Coef. Std. Err. t P>|t| [95% Conf. Interval]

he | 12534.92 45987.93 0.27 0.785 -77859.64 102929.5
we | 18190.56 68279.11 0.27 0.790 -116019.9 152401
kl6 | -56323.29 205150.6 -0.27 0.784 -459570.4 346923.8
faminc_hat2 | -.0000935 .0002698 -0.35 0.729 -.0006237 .0004368
faminc_hat3 | 9.41e-10 2.18e-09 0.43 0.666 -3.34e-09 5.22e-09
faminc_hat4 | -3.02e-15 6.38e-15 -0.47 0.637 -1.56e-14 9.53e-15
_cons | -20503.47 383113.9 -0.05 0.957 -773557.8 732550.9



test faminc_hat2 faminc_hat3 faminc_hat4

( 1) faminc_hat2 = 0
( 2) faminc_hat3 = 0
( 3) faminc_hat4 = 0
Constraint 2 dropped
Constraint 3 dropped

F( 1, 421) = 0.12
Prob > F = 0.7292

[Richard Williams]

Thanks Simon. Again, it would be far easier to read this if you used code tags. See pt 12 of the FAQ.

[Simon Bensnes]

Richard, Hope this is better.

Variable | Obs Mean Std. Dev. Min Max

faminc | 428 91213 44117.35 9072 344146.3
he | 428 12.61215 3.035163 4 17
we | 428 12.65888 2.285376 5 17
kl6 | 428 .1401869 .3919231 0 2



Source | SS df MS Number of obs = 428
F( 3, 424) = 30.43
Model | 1.4725e+11 3 4.9082e+10 Prob > F = 0.0000
Residual | 6.8384e+11 424 1.6128e+09 R-squared = 0.1772
Adj R-squared = 0.1714
Total | 8.3109e+11 427 1.9463e+09 Root MSE = 40160


faminc | Coef. Std. Err. t P>|t| [95% Conf. Interval]

he | 3211.526 796.7026 4.03 0.000 1645.547 4777.504
we | 4776.907 1061.164 4.50 0.000 2691.111 6862.704
kl6 | -14310.92 5003.928 -2.86 0.004 -24146.52 -4475.326
_cons | -7755.33 11162.93 -0.69 0.488 -29696.91 14186.25


. estat ovtest

Ramsey RESET test using powers of the fitted values of faminc
Ho: model has no omitted variables
F(3, 421) = 2.15
Prob > F = 0.0931


Source | SS df MS Number of obs = 428
F( 6, 421) = 16.42
Model | 1.5758e+11 6 2.6263e+10 Prob > F = 0.0000
Residual | 6.7351e+11 421 1.5998e+09 R-squared = 0.1896
Adj R-squared = 0.1781
Total | 8.3109e+11 427 1.9463e+09 Root MSE = 39997


faminc | Coef. Std. Err. t P>|t| [95% Conf. Interval]

he | 12534.92 45987.93 0.27 0.785 -77859.64 102929.5
we | 18190.56 68279.11 0.27 0.790 -116019.9 152401
kl6 | -56323.29 205150.6 -0.27 0.784 -459570.4 346923.8
faminc_hat2 | -.0000935 .0002698 -0.35 0.729 -.0006237 .0004368
faminc_hat3 | 9.41e-10 2.18e-09 0.43 0.666 -3.34e-09 5.22e-09
faminc_hat4 | -3.02e-15 6.38e-15 -0.47 0.637 -1.56e-14 9.53e-15
_cons | -20503.47 383113.9 -0.05 0.957 -773557.8 732550.9



test faminc_hat2 faminc_hat3 faminc_hat4

( 1) faminc_hat2 = 0
( 2) faminc_hat3 = 0
( 3) faminc_hat4 = 0
Constraint 2 dropped
Constraint 3 dropped

F( 1, 421) = 0.12
Prob > F = 0.7292

[Richard Williams]

Even though you are using code tags, it still looks screwed up. Make sure you are copying and pasting from the original Stata output.

Also, you aren't showing your predict and generate commands. Make sure you are using double precision. For example:

. webuse nhanes2f, clear

. reg weight height age

      Source |       SS           df       MS      Number of obs   =    10,337
-------------+----------------------------------   F(2, 10334)     =   1702.94
       Model |  604097.713         2  302048.857   Prob > F        =    0.0000
    Residual |  1832929.54    10,334  177.368835   R-squared       =    0.2479
-------------+----------------------------------   Adj R-squared   =    0.2477
       Total |  2437027.25    10,336    235.7805   Root MSE        =    13.318

------------------------------------------------------------------------------
      weight |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      height |   .8063681   .0138592    58.18   0.000     .7792015    .8335348
         age |   .1279232   .0077761    16.45   0.000     .1126805    .1431659
       _cons |  -69.37224   2.430505   -28.54   0.000     -74.1365   -64.60798
------------------------------------------------------------------------------

. ovtest

Ramsey RESET test using powers of the fitted values of weight
       Ho:  model has no omitted variables
               F(3, 10331) =     11.02
                  Prob > F =      0.0000

. predict double yhat
(option xb assumed; fitted values)

. gen double yhat2 = yhat ^2

. gen double yhat3 = yhat ^3

. gen double yhat4 = yhat ^4

. reg weight height age yhat2 yhat3 yhat4

      Source |       SS           df       MS      Number of obs   =    10,337
-------------+----------------------------------   F(5, 10331)     =    689.77
       Model |  609944.961         5  121988.992   Prob > F        =    0.0000
    Residual |  1827082.29    10,331   176.85435   R-squared       =    0.2503
-------------+----------------------------------   Adj R-squared   =    0.2499
       Total |  2437027.25    10,336    235.7805   Root MSE        =    13.299

------------------------------------------------------------------------------
      weight |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      height |   8.304195   15.40679     0.54   0.590     -21.8961    38.50449
         age |   1.319286   2.444151     0.54   0.589    -3.471722    6.110295
       yhat2 |  -.2557669   .4030469    -0.63   0.526    -1.045817    .5342832
       yhat3 |    .002833   .0037523     0.75   0.450    -.0045223    .0101883
       yhat4 |   -.000011    .000013    -0.85   0.396    -.0000365    .0000145
       _cons |  -819.4836   1662.432    -0.49   0.622    -4078.173    2439.205
------------------------------------------------------------------------------

. test yhat2 yhat3 yhat4

 ( 1)  yhat2 = 0
 ( 2)  yhat3 = 0
 ( 3)  yhat4 = 0

       F(  3, 10331) =   11.02
            Prob > F =    0.0000

.

[Simon Bensnes]

I see. I am sorry. I copied the text "as table". It seems to be correct now.

. sum

    Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
      faminc |       428       91213    44117.35       9072   344146.3
          he |       428    12.61215    3.035163          4         17
          we |       428    12.65888    2.285376          5         17
         kl6 |       428    .1401869    .3919231          0          2


      Source |       SS       df       MS              Number of obs =     428
-------------+------------------------------           F(  3,   424) =   30.43
       Model |  1.4725e+11     3  4.9082e+10           Prob > F      =  0.0000
    Residual |  6.8384e+11   424  1.6128e+09           R-squared     =  0.1772
-------------+------------------------------           Adj R-squared =  0.1714
       Total |  8.3109e+11   427  1.9463e+09           Root MSE      =   40160

------------------------------------------------------------------------------
      faminc |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
          he |   3211.526   796.7026     4.03   0.000     1645.547    4777.504
          we |   4776.907   1061.164     4.50   0.000     2691.111    6862.704
         kl6 |  -14310.92   5003.928    -2.86   0.004    -24146.52   -4475.326
       _cons |   -7755.33   11162.93    -0.69   0.488    -29696.91    14186.25
------------------------------------------------------------------------------

. estat ovtest

Ramsey RESET test using powers of the fitted values of faminc
       Ho:  model has no omitted variables
                 F(3, 421) =      2.15
                  Prob > F =      0.0931

. 
. reg faminc he we kl6

      Source |       SS       df       MS              Number of obs =     428
-------------+------------------------------           F(  3,   424) =   30.43
       Model |  1.4725e+11     3  4.9082e+10           Prob > F      =  0.0000
    Residual |  6.8384e+11   424  1.6128e+09           R-squared     =  0.1772
-------------+------------------------------           Adj R-squared =  0.1714
       Total |  8.3109e+11   427  1.9463e+09           Root MSE      =   40160

------------------------------------------------------------------------------
      faminc |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
          he |   3211.526   796.7026     4.03   0.000     1645.547    4777.504
          we |   4776.907   1061.164     4.50   0.000     2691.111    6862.704
         kl6 |  -14310.92   5003.928    -2.86   0.004    -24146.52   -4475.326
       _cons |   -7755.33   11162.93    -0.69   0.488    -29696.91    14186.25
------------------------------------------------------------------------------

. predict faminc_hat
(option xb assumed; fitted values)

. gen faminc_hat2 = faminc_hat^2

. gen faminc_hat3 = faminc_hat^3

. gen faminc_hat4 = faminc_hat^4

. 
. reg faminc he we kl6 faminc_hat2 faminc_hat3 faminc_hat4

      Source |       SS       df       MS              Number of obs =     428
-------------+------------------------------           F(  6,   421) =   16.42
       Model |  1.5758e+11     6  2.6263e+10           Prob > F      =  0.0000
    Residual |  6.7351e+11   421  1.5998e+09           R-squared     =  0.1896
-------------+------------------------------           Adj R-squared =  0.1781
       Total |  8.3109e+11   427  1.9463e+09           Root MSE      =   39997

------------------------------------------------------------------------------
      faminc |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
          he |   12534.92   45987.93     0.27   0.785    -77859.64    102929.5
          we |   18190.56   68279.11     0.27   0.790    -116019.9      152401
         kl6 |  -56323.29   205150.6    -0.27   0.784    -459570.4    346923.8
 faminc_hat2 |  -.0000935   .0002698    -0.35   0.729    -.0006237    .0004368
 faminc_hat3 |   9.41e-10   2.18e-09     0.43   0.666    -3.34e-09    5.22e-09
 faminc_hat4 |  -3.02e-15   6.38e-15    -0.47   0.637    -1.56e-14    9.53e-15
       _cons |  -20503.47   383113.9    -0.05   0.957    -773557.8    732550.9
------------------------------------------------------------------------------

. test faminc_hat2 faminc_hat3 faminc_hat4

 ( 1)  faminc_hat2 = 0
 ( 2)  faminc_hat3 = 0
 ( 3)  faminc_hat4 = 0
       Constraint 2 dropped
       Constraint 3 dropped

       F(  1,   421) =    0.12
            Prob > F =    0.7292

[Richard Williams]

Again, use double precision, like I did in my example.

[Richard Williams]

If you look at the code for ovtest.ado, you see that it too uses double precision for both the predict and gen commands.

[Simon Bensnes]

Richard,

I tried to use the double precision now (both for gen and for predict), it does not solve the problem unfortunately.

[Carlo Lazzaro]

Dear All,
strange as it seems, I can replicate Simon's problem with -auto.dta-:

. sysuse auto.dta
(1978 Automobile Data)

. reg price mpg

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(1, 72)        =     20.26
       Model |   139449474         1   139449474   Prob > F        =    0.0000
    Residual |   495615923        72  6883554.48   R-squared       =    0.2196
-------------+----------------------------------   Adj R-squared   =    0.2087
       Total |   635065396        73  8699525.97   Root MSE        =    2623.7

------------------------------------------------------------------------------
       price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         mpg |  -238.8943   53.07669    -4.50   0.000    -344.7008   -133.0879
       _cons |   11253.06   1170.813     9.61   0.000     8919.088    13587.03
------------------------------------------------------------------------------

. estat ovtest

Ramsey RESET test using powers of the fitted values of price
       Ho:  model has no omitted variables
                  F(3, 69) =      7.72
                  Prob > F =      0.0002

. predict price_hat
(option xb assumed; fitted values)

. gen double price_hat2=price_hat^2

. gen double price_hat3=price_hat^3

. gen double price_hat4=price_hat^4


. reg price mpg price_hat2 price_hat3 price_hat4

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(4, 69)        =     12.27
       Model |   263997537         4  65999384.3   Prob > F        =    0.0000
    Residual |   371067859        69  5377795.06   R-squared       =    0.4157
-------------+----------------------------------   Adj R-squared   =    0.3818
       Total |   635065396        73  8699525.97   Root MSE        =      2319

------------------------------------------------------------------------------
       price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         mpg |   4321.776   2876.027     1.50   0.137     -1415.74    10059.29
  price_hat2 |   .0067004   .0040298     1.66   0.101    -.0013388    .0147396
  price_hat3 |  -1.02e-06   5.56e-07    -1.83   0.072    -2.13e-06    9.20e-08
  price_hat4 |   5.52e-11   2.71e-11     2.04   0.046     1.12e-12    1.09e-10
       _cons |  -182925.1   123551.7    -1.48   0.143      -429404    63553.85
------------------------------------------------------------------------------

. test price_hat2 price_hat3 price_hat4

 ( 1)  price_hat2 = 0
 ( 2)  price_hat3 = 0
 ( 3)  price_hat4 = 0
       Constraint 2 dropped
       Constraint 3 dropped

       F(  1,    69) =    2.76
            Prob > F =    0.1009

[Joao Santos Silva]

Dear All,

This is a problem I am very familiar with (I am a great fan of the RESET!) and even last week I used this in class to illustrate the kind of numerical issues that we often face.

The problem is caused by near perfect collienarity between the powers of the fitted values when the mean of y is high and x varies little. Generally I avoid the problem by centering the fitted values at zero before constructing the powers but in Carlo's example that did not work. An alternative that works there is simply to work with the price in thousands (that is, divide price by 1000 before doing the regression).

Simon, please do let us know if any of these tricks works in your example.

All the best,

Joao

[Simon Bensnes]

Joao,

Dividing by 1000 worked swimmingly. Thank you very much.

Carlo: Thank you for your time replicating my example in auto.dta set. I did not try that specific combination of variables.

[Richard Williams]

Thanks Joao. I suspected as much, as my own tests were showing super-high correlations between the powers. I wonder how ovtest avoids this problem? Perhaps if we looked at the code more carefully, we would see it is employing tricks such as you suggest.

[Carlo Lazzaro]

Thanks everybody for this interesting thread.