Autoregressive Least Squares (ALS) - Cochrane - Orcutt Regression

Emad Shehata

Join Date: Oct 2014
Posts: 203

Autoregressive Least Squares (ALS) - Cochrane - Orcutt Regression

10 May 2016, 05:37

I will Introduce here some examples for Autoregressive Least Squares (ALS) models
and especially Cochrane - Orcutt Regression via:
1- ALS - Autoregressive Least Squares transformations.
2- NLS - Nonlinear Least Squares Estimation
3- MLE - Maximum likelihood Estimation

Code:

clear all
 input t y x1 x2
 1  99.2  96.7   101
 2    99  98.1 100.1
 3   100   100   100
 4 111.6 104.9  90.6
 5 122.2 104.9  86.5
 6 117.6 109.5  89.7
 7 121.1 110.8  90.6
 8   136 112.3  82.8
 9 154.2 109.3  70.1
10 153.6 105.3  65.4
11 158.5 101.7  61.3
12 140.6  95.4  62.5
13 136.2  96.4  63.6
14   168  97.6  52.6
15 154.3 102.4  59.7
16   149 101.6  59.5
17 165.5 103.8  61.3
 end

 tsset t
 gen y1=L.y
 gen x11=L.x1
 gen x21=L.x2

* ALS - Cochrane-Orcutt Regression

Code:

 prais y x1 x2 , corc
predict double e , res
reg e L.e , noconst

Code:

drop in 1

* NLS - Cochrane-Orcutt Regression

Code:

nl (y=({B1}*x1+{B2}*x2+{B0}-{R}*({B1}*x11+{B2}*x21+{B0}))+{R}*y1)

* MLE - Cochrane-Orcutt Regression

Code:

program define mle1
 args lnf B0 B1 B2 R S
  qui replace `lnf'= -0.5*ln(2*_pi*`S'^2)-(1/(2*`S'^2))*(($ML_y1-`B0'-`B1'*x1-`B2'*x2) -`R'*(y1-`B0'-`B1'*x11-`B2'*x21))^2
 end
 ml model lf mle1 (B0:y=) (B1:) (B2:) (Rho:) (Sigma:) , technique(nr)
 ml init 120 1.2 -1.4 0  5.5, copy
 ml maximize

* MLE - Cochrane-Orcutt Regression

Code:

program define mle2
 args lnf B0 B1 B2 R S
 qui replace `lnf'= -0.5*ln(2*_pi*`S'^2)-0.5*($ML_y1-`R'*y1-`B0'+`R'*`B0' -`B1'*x1+`R'*`B1'*x11 -`B2'*x2+`R'*`B2'*x21)^2/`S'^2
 end
 ml model lf mle2 (B0:y=) (B1:) (B2:) (Rho:) (Sigma:) , technique(nr)
 ml init 120 1.2 -1.4 0  5.5, copy
 ml maximize

HTML Code:

.  prais y x1 x2 , corc

Iteration 0:  rho = 0.0000
Iteration 1:  rho = -0.1824
Iteration 2:  rho = -0.1603
Iteration 3:  rho = -0.1564
Iteration 4:  rho = -0.1557
Iteration 5:  rho = -0.1556
Iteration 6:  rho = -0.1555
Iteration 7:  rho = -0.1555
Iteration 8:  rho = -0.1555

Cochrane-Orcutt AR(1) regression -- iterated estimates

      Source |       SS       df       MS              Number of obs =      16
-------------+------------------------------           F(  2,    13) =  160.30
       Model |   9404.0186     2   4702.0093           Prob > F      =  0.0000
    Residual |  381.325374    13  29.3327211           R-squared     =  0.9610
-------------+------------------------------           Adj R-squared =  0.9550
       Total |  9785.34397    15  652.356265           Root MSE      =   5.416

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
          x1 |   1.168129   .2487314     4.70   0.000     .6307779    1.705481
          x2 |  -1.413293   .0790578   -17.88   0.000    -1.584087   -1.242499
       _cons |   121.6113   24.50442     4.96   0.000     68.67271    174.5499
-------------+----------------------------------------------------------------
         rho |  -.1555319
------------------------------------------------------------------------------
Durbin-Watson statistic (original)    2.018549
Durbin-Watson statistic (transformed) 2.027891

.  predict double e , res
.  reg e L.e , noconst

      Source |       SS       df       MS              Number of obs =      16
-------------+------------------------------           F(  1,    15) =    0.34
       Model |  8.66958701     1  8.66958701           Prob > F      =  0.5679
    Residual |  381.325374    15  25.4216916           R-squared     =  0.0222
-------------+------------------------------           Adj R-squared = -0.0430
       Total |  389.994961    16  24.3746851           Root MSE      =   5.042

------------------------------------------------------------------------------
           e |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           e |
         L1. |  -.1555317   .2663312    -0.58   0.568    -.7232032    .4121397
------------------------------------------------------------------------------

.  drop in 1
(1 observation deleted)

. * NLS - Cochrane-Orcutt Regression

.  nl (y=({B1}*x1+{B2}*x2+{B0}-{R}*({B1}*x11+{B2}*x21+{B0}))+{R}*y1)
(obs = 16)

Iteration 0:  residual SS =  9926.351
Iteration 1:  residual SS =  424.7784
Iteration 2:  residual SS =  381.5585
Iteration 3:  residual SS =  381.3257
Iteration 4:  residual SS =  381.3254
Iteration 5:  residual SS =  381.3254
Iteration 6:  residual SS =  381.3254
Iteration 7:  residual SS =  381.3254

      Source |       SS       df       MS
-------------+------------------------------         Number of obs =        16
       Model |  7188.51265     3  2396.17088         R-squared     =    0.9496
    Residual |  381.325374    12  31.7771145         Adj R-squared =    0.9370
-------------+------------------------------         Root MSE      =  5.637119
       Total |  7569.83803    15  504.655869         Res. dev.     =  96.14306

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         /B1 |   1.168129   .2705726     4.32   0.001     .5786022    1.757657
         /B2 |  -1.413293   .0881198   -16.04   0.000    -1.605289   -1.221296
         /B0 |   121.6113   26.14065     4.65   0.001     64.65571    178.5669
          /R |  -.1555315   .3240643    -0.48   0.640     -.861607    .5505439
------------------------------------------------------------------------------
  Parameter B0 taken as constant term in model & ANOVA table

. * MLE - Cochrane-Orcutt Regression

.  program define mle1
.  args lnf B0 B1 B2 R S
.   qui replace `lnf'= -0.5*ln(2*_pi*`S'^2)-(1/(2*`S'^2))*(($ML_y1-`B0'-`B1'*x1-`B2'*x2) -`R'*(y1-`B0'-`B1'* x11-`B2'*x21))^2
.  end
.  ml model lf mle1 (B0:y=) (B1:) (B2:) (Rho:) (Sigma:) , technique(nr)
.  ml init 120 1.2 -1.4 0  5.5, copy
.  ml maximize

initial:       log likelihood = -50.346957
rescale:       log likelihood = -50.346957
rescale eq:    log likelihood = -50.346957
Iteration 0:   log likelihood = -50.346957  (not concave)
Iteration 1:   log likelihood = -48.197837  (not concave)
Iteration 2:   log likelihood = -48.080684 
Iteration 3:   log likelihood = -48.071543 
Iteration 4:   log likelihood = -48.071531 
Iteration 5:   log likelihood = -48.071531 

                                                  Number of obs   =         16
                                                  Wald chi2(0)    =          .
Log likelihood = -48.071531                       Prob > chi2     =          .

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
B0           |
       _cons |   121.6113   22.36593     5.44   0.000     77.77486    165.4477
-------------+----------------------------------------------------------------
B1           |
       _cons |   1.168129   .2316487     5.04   0.000     .7141063    1.622152
-------------+----------------------------------------------------------------
B2           |
       _cons |  -1.413293   .0785321   -18.00   0.000    -1.567213   -1.259373
-------------+----------------------------------------------------------------
Rho          |
       _cons |  -.1555317   .2859157    -0.54   0.586    -.7159162    .4048528
-------------+----------------------------------------------------------------
Sigma        |
       _cons |   4.881889   .8630042     5.66   0.000     3.190432    6.573346
------------------------------------------------------------------------------
.
. * MLE - Cochrane-Orcutt Regression

.  program define mle2
.  args lnf B0 B1 B2 R S
.  qui replace `lnf'= -0.5*ln(2*_pi*`S'^2)-0.5*($ML_y1-`R'*y1-`B0'+`R'*`B0' -`B1'*x1+`R'*`B1'*x11 -`B2'*x2+`R'*`B2'*x21)^2/`S'^2
.  end
.  ml model lf mle2 (B0:y=) (B1:) (B2:) (Rho:) (Sigma:) , technique(nr)
.  ml init 120 1.2 -1.4 0  5.5, copy
.  ml maximize

initial:       log likelihood = -50.346957
rescale:       log likelihood = -50.346957
rescale eq:    log likelihood = -50.346957
Iteration 0:   log likelihood = -50.346957  (not concave)
Iteration 1:   log likelihood = -48.197755  (not concave)
Iteration 2:   log likelihood = -48.080669 
Iteration 3:   log likelihood = -48.071543 
Iteration 4:   log likelihood = -48.071531 
Iteration 5:   log likelihood = -48.071531 

                                                  Number of obs   =         16
                                                  Wald chi2(0)    =          .
Log likelihood = -48.071531                       Prob > chi2     =          .

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
B0           |
       _cons |   121.6113   22.36589     5.44   0.000     77.77496    165.4476
-------------+----------------------------------------------------------------
B1           |
       _cons |   1.168129   .2316482     5.04   0.000     .7141074    1.622151
-------------+----------------------------------------------------------------
B2           |
       _cons |  -1.413293    .078532   -18.00   0.000    -1.567213   -1.259373
-------------+----------------------------------------------------------------
Rho          |
       _cons |  -.1555317   .2859157    -0.54   0.586    -.7159161    .4048528
-------------+----------------------------------------------------------------
Sigma        |
       _cons |   4.881889   .8630042     5.66   0.000     3.190432    6.573346
------------------------------------------------------------------------------

Emad A. Shehata
Professor (PhD Economics)
Agricultural Research Center - Agricultural Economics Research Institute - Egypt
Email: [email protected]
IDEAS: http://ideas.repec.org/f/psh494.html
EconPapers: http://econpapers.repec.org/RAS/psh494.htm
Google Scholar: http://scholar.google.com/citations?...r=cOXvc94AAAAJ

Tags: None

Emad Shehata

Join Date: Oct 2014
Posts: 203

10 May 2016, 05:42

HTML Code:

. arima y x1 x2 , ar(1)

(setting optimization to BHHH)
Iteration 0:   log likelihood = -51.415604 
Iteration 1:   log likelihood = -51.401997 
Iteration 2:   log likelihood = -51.400804 
Iteration 3:   log likelihood = -51.399342 
Iteration 4:   log likelihood =  -51.39834 
(switching optimization to BFGS)
Iteration 5:   log likelihood = -51.397883 
Iteration 6:   log likelihood = -51.397162 
Iteration 7:   log likelihood = -51.397126 
Iteration 8:   log likelihood = -51.397125 
Iteration 9:   log likelihood = -51.397124 

ARIMA regression

Sample:  1 - 17                                 Number of obs      =        17
                                                Wald chi2(3)       =    365.59
Log likelihood = -51.39712                      Prob > chi2        =    0.0000

------------------------------------------------------------------------------
             |                 OPG
           y |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
y            |
          x1 |   1.065039   .2618857     4.07   0.000     .5517522    1.578325
          x2 |  -1.375039   .0854707   -16.09   0.000    -1.542558   -1.207519
       _cons |   129.6103   26.79329     4.84   0.000      77.0964    182.1241
-------------+----------------------------------------------------------------
ARMA         |
          ar |
         L1. |  -.1979814   .3781801    -0.52   0.601    -.9392007    .5432379
-------------+----------------------------------------------------------------
      /sigma |   4.969051   1.417814     3.50   0.000     2.190187    7.747914
------------------------------------------------------------------------------

Comment

Emad Shehata

Join Date: Oct 2014
Posts: 203

10 May 2016, 05:46

with arima of order AR(1)
is identical to Beach-Mackinnon AR(1) Autoregressive Maximum Likelihood Estimation
as follows:

Code:

. alsmle y x1 x2 , dn
=========================================================================
* Beach-Mackinnon AR(1) Autoregressive Maximum Likelihood Estimation
=========================================================================
  Number of Obs    =         17
  Wald Test        =   457.1243         P-Value > Chi2(2)        =     0.0000
  F Test           =   228.5621         P-Value > F(2 , 17)      =     0.0000
  R-squared        =     0.9528         Raw Moments R2           =     0.9987
  R-squared Adj    =     0.9556         Raw Moments R2 Adj       =     0.9988
  Root MSE (Sigma) =     4.9691         Log Likelihood Function  =   -51.3971
  Autoregressive Coefficient (Rho) Value = -0.1979982
------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
          x1 |   1.065033   .2067004     5.15   0.000      .628933    1.501133
          x2 |  -1.375032   .0643558   -21.37   0.000     -1.51081   -1.239253
       _cons |   129.6105   20.87587     6.21   0.000     85.56626    173.6547
------------------------------------------------------------------------------

Announcement

Autoregressive Least Squares (ALS) - Cochrane - Orcutt Regression

Comment

Comment