Dealing with Highly Autocorrelated Time Series data

[Guest]

Dear all,

I am facing an issue here - my data is obtained from the stock market and since time series data has tendency to be autocorrelated, I decided to test if that's the case using two tests, namely the DW test and also the BG test. The results were consistent, indicating autocorrelation, if the order of autocorrelation is something like 1 or 2, it could easily be dealt with specifying an AR(1) or AR(2) model, however, as you can see below, the lags were way more significant than that. What does this phenomenon imply, and what can I possibly do about it?

. tsset date
        time variable:  date, 01jan2001 to 31dec2010, but with gaps
                delta:  1 day

. dwstat

Number of gaps in sample:  521

Durbin-Watson d-statistic(  3,  2609) =  .7229313

. estat bgodfrey, lag(1)

Number of gaps in sample:  521

Breusch-Godfrey LM test for autocorrelation
---------------------------------------------------------------------------
    lags(p)  |          chi2               df                 Prob > chi2
-------------+-------------------------------------------------------------
       1     |        636.538               1                   0.0000
---------------------------------------------------------------------------
                        H0: no serial correlation

. estat bgodfrey, lag(2)

Number of gaps in sample:  521

Breusch-Godfrey LM test for autocorrelation
---------------------------------------------------------------------------
    lags(p)  |          chi2               df                 Prob > chi2
-------------+-------------------------------------------------------------
       2     |        695.457               2                   0.0000
---------------------------------------------------------------------------
                        H0: no serial correlation

. estat bgodfrey, lag(50)

Number of gaps in sample:  521

Breusch-Godfrey LM test for autocorrelation
---------------------------------------------------------------------------
    lags(p)  |          chi2               df                 Prob > chi2
-------------+-------------------------------------------------------------
      50     |       1008.710              50                   0.0000
---------------------------------------------------------------------------
                        H0: no serial correlation

. estat bgodfrey, lag(100)

Number of gaps in sample:  521

Breusch-Godfrey LM test for autocorrelation
---------------------------------------------------------------------------
    lags(p)  |          chi2               df                 Prob > chi2
-------------+-------------------------------------------------------------
     100     |       1050.282             100                   0.0000
---------------------------------------------------------------------------
                        H0: no serial correlation

Below is my example data, for all it's worth:

* Example generated by -dataex-. To install: ssc install dataex
clear
input int date float(csad r_mt abs_r_mt rmt2)
14976         .          .         .          .
14977  1.570728  -.3534996  .3534996  .12496194
14978 2.1024446  -.7965839  .7965839   .6345459
14979  2.229388  1.5501753 1.5501753  2.4030435
14980  .7713282 -.51517767 .51517767  .26540804
14983 2.1556273  -2.815203  2.815203   7.925369
14984 1.4671476  -.7154922  .7154922  .51192915
14985 2.2644396  3.4870846 3.4870846   12.15976
14986 1.5703444  -.9405673  .9405673   .8846669
14987 1.1591344 -.26794058 .26794058 .071792156
14990 1.9483218  -2.778993  2.778993   7.722802
14991 1.8586833  -2.469931  2.469931   6.100557
14992  .7667049  -.3789219  .3789219  .14358181
14993  1.560813  1.3359466 1.3359466  1.7847532
14994  1.683192  1.1070224 1.1070224  1.2254986
14997         0          0         0          0
14998         0          0         0          0
14999         0          0         0          0
15000         0          0         0          0
15001         0          0         0          0
15004         0          0         0          0
15005         0          0         0          0
15006         0          0         0          0
15007         0          0         0          0
15008         0          0         0          0
15011 2.0803628   -3.05803   3.05803   9.351548
15012  .9748077  .04337036 .04337036 .001880988
15013 1.2678893 -.58709294 .58709294   .3446781
15014  1.604526 -4.3798285 4.3798285    19.1829
15015  .9917876   .2995471  .2995471  .08972847
15018  1.569664  -.8122451  .8122451   .6597421
15019  .8885421   .3391319  .3391319  .11501046
15020 1.7289377  1.1798655 1.1798655  1.3920826
15021  1.469052 -.31126085 .31126085  .09688332
15022 1.4224944   .9744703  .9744703   .9495923
15025 1.6597908   3.510377  3.510377  12.322746
15026         0          0         0          0
15027         0          0         0          0
15028         0          0         0          0
15029         0          0         0          0
15032         0          0         0          0
15033         0          0         0          0
15034 1.2572125   9.423772  9.423772   88.80747
15035  1.275415   9.445167  9.445167   89.21117
15036 1.1821964   9.417044  9.417044   88.68071
15039    .83358   9.437843  9.437843   89.07288
15040 3.4699204   4.943833  4.943833   24.44149
15041 4.2142005   -6.86108   6.86108   47.07442
15042 2.3196154  .24586634 .24586634  .06045026
15043  1.494033   7.036363  7.036363    49.5104
15046 2.9070094   4.637869  4.637869   21.50983
15047  3.157141 -1.5542407 1.5542407   2.415664
15048  2.025874  2.5596695 2.5596695   6.551908
15049 1.9587705  -1.498094  1.498094  2.2442858
15050 1.5076005   -.870127   .870127    .757121
15053 1.7753855   3.826742  3.826742  14.643955
15054 1.7674456  .56592834 .56592834   .3202749
15055  1.951145  2.3093104 2.3093104   5.332915
15056  1.836256   6.192546  6.192546   38.34763
15057 2.4204204  -8.198447  8.198447   67.21454
15060 1.1888419    8.16455   8.16455  66.659874
15061  1.936496    3.23113   3.23113    10.4402
15062 1.9047937   .8242897  .8242897   .6794535
15063  2.256077  -.9183025  .9183025   .8432794
15064   1.29831   1.059913  1.059913  1.1234157
15067 1.5063257    2.09295   2.09295    4.38044
15068 1.5023632   .4708089  .4708089    .221661
15069  1.532801 -2.2716038 2.2716038   5.160184
15070 1.1850337   1.204754  1.204754   1.451432
15071 1.4514213  1.1289606 1.1289606   1.274552
15074  1.201572  1.7730404 1.7730404   3.143672
15075 1.1211928  1.2764815 1.2764815   1.629405
15076  1.314867  1.8994558 1.8994558   3.607932
15077 1.2817875   .9410506  .9410506   .8855762
15078 1.4225343   .9015344  .9015344   .8127642
15081 1.4764158   2.912991  2.912991   8.485517
15082 1.5169854  2.3605452 2.3605452   5.572174
15083 1.4291006  2.2689912 2.2689912   5.148321
15084 2.2791367  .57240033 .57240033   .3276421
15085 1.4125174 -.24851386 .24851386  .06175914
15088 2.0327883 -1.4182184 1.4182184  2.0113432
15089  1.355204  3.8184226 3.8184226   14.58035
15090 1.3968363  -2.830132  2.830132   8.009648
15091 1.3878628 -2.1431394 2.1431394   4.593046
15092 1.6446024  -2.867832  2.867832   8.224462
15095 1.1272126  2.7186074 2.7186074   7.390826
15096         0          0         0          0
15097         0          0         0          0
15098         0          0         0          0
15099         0          0         0          0
15102         0          0         0          0
15103     1.463   5.472131  5.472131   29.94422
15104  .9655043 -.14305109 .14305109 .020463614
15105 1.3336984  2.0841575 2.0841575  4.3437123
15106 1.2852207  1.2281387 1.2281387  1.5083246
15109 1.1468617   1.713324  1.713324   2.935479
15110 1.1242508   .7291949  .7291949  .53172517
15111 1.3639222   1.700456  1.700456    2.89155
15112 1.1554041   .5353749  .5353749  .28662625
15113  1.136401    .766858   .766858  .58807117
end
format %td date

[Filippo Busetto]

If you have autocorrelation with stock market data, I assume you are working with non stationary data. Are you using prices instead of returns?

[Guest]

Dear Filippo,

I have carried out a DF test and I think my data is stationary, and I am using the market returns, not prices.

. dfuller rmt2, trend lags(0)

Dickey-Fuller test for unit root                   Number of obs   =      2087

                               ---------- Interpolated Dickey-Fuller ---------
                  Test         1% Critical       5% Critical      10% Critical
               Statistic           Value             Value             Value
------------------------------------------------------------------------------
 Z(t)            -48.559            -3.960            -3.410            -3.120
------------------------------------------------------------------------------
MacKinnon approximate p-value for Z(t) = 0.0000

. dfuller rmt2, trend lags(1)

Augmented Dickey-Fuller test for unit root         Number of obs   =      1565

                               ---------- Interpolated Dickey-Fuller ---------
                  Test         1% Critical       5% Critical      10% Critical
               Statistic           Value             Value             Value
------------------------------------------------------------------------------
 Z(t)            -29.892            -3.960            -3.410            -3.120
------------------------------------------------------

https://www.statalist.org/forums/for...-herd-behavior <- if you want some background of my project, here it is

[Filippo Busetto]

I would start by assessing all the zeros in your dataset, which i think creates some major issues. what are those?

It lloks very strange to me that retruns would be this much serially correlated. Anyway, usually the simplest way to solve it would be to use Newey west standard errors in your regression (which seems to be the standard approach in asset pricing nowadays).

[Guest]

Dear Filippo,

I can assure you that those zeros are innocuous. The zero is derived from subtracting stock price at time t-1 from stock price at time t, hence, if stock A costs $1 today and tomorrow, the price difference, which, in my model I assume is the 'return' in a simplified way, is going to be zero.

Thank you for your input, as far as I am concerned, newey-west standard errors correct autocorrelation and heteroskedasticity, right? May I know how do I execute the -newey- command in Stata, and what lags should I choose?

[David Clark]

Dear Filippo,

I have carried out a DF test and I think my data is stationary, and I am using the market returns, not prices.

. dfuller rmt2, trend lags(0)

Dickey-Fuller test for unit root Number of obs = 2087

---------- Interpolated Dickey-Fuller ---------
Test 1% Critical 5% Critical 10% Critical
Statistic Value Value Value
------------------------------------------------------------------------------
Z(t) -48.559 -3.960 -3.410 -3.120
------------------------------------------------------------------------------
MacKinnon approximate p-value for Z(t) = 0.0000

. dfuller rmt2, trend lags(1)

Augmented Dickey-Fuller test for unit root Number of obs = 1565

---------- Interpolated Dickey-Fuller ---------
Test 1% Critical 5% Critical 10% Critical
Statistic Value Value Value
------------------------------------------------------------------------------
Z(t) -29.892 -3.960 -3.410 -3.120
------------------------------------------------------

https://www.statalist.org/forums/for...-herd-behavior <- if you want some background of my project, here it is

Guest, your implementation of the ADF isn't correct. You need to include enough differenced lags of the variable into the ADF specification, to account for the serial correlation, in order to be able to calculate a test statistic that is interpretable.

[Guest]

Dear David,

My apologies if I were to sound ignorant to you, but what is 'enough' and what is not? Is it to do with Information Criteria regarding lags selection?

Thank you.

[Filippo Busetto]

In order to check for lag lenght for newey west, I would start by looking at the partial autocorrelation function. Anyway, those lines of zero still seem pretty dodgy to me. If those are not trading days i would get rid of those before doing anything.