How to use xtcointtest with runby? OR How to do cointegration tests for each individuals separately for panel data?

[Qingshi Wang]

Dear all:
I'm trying to use xtcointtest to do cointegration tests for each individuals separately in panel data and store the results, so I need run the command xtcointtest by groups. However I failed by using the command runby.
My question is: why dose this error occur? and how to solve it?

the example data:

input float(pair ym mcprice2) double mcprice3
1 683      3395            3712.5
1 682      3058              3160
1 681    2847.5              2800
1 680      2800            2762.5
1 679      2782              2870
1 678      2560              2725
1 677      2414              2580
1 676    2562.5              2700
1 675      2610              2500
1 674      2282              2550
1 673    1962.5              2250
1 672      1940              2300
1 671      1932              2230
1 670      2085            2262.5
1 669      2160              2400
1 668      2174              2520
1 667 2251.6667              2600
1 666      2260              2670
1 665      2395            2837.5
1 664   2453.75              2875
1 663      2476              2930
1 662      2480              2925
1 661      2510              2900
1 660      2725            2962.5
1 659      2903              3000
1 658   2888.75              3325
1 657      2910              3325
1 656   3043.75              3325
1 655    3132.5              3325
1 624      4400              4600
1 653      3200              3325
1 652    3287.5              3650
1 651      3398              3650
1 650      3410            3687.5
1 649   3526.25              3700
1 648      3567              3700
1 647      3620              3695
1 646   3603.75            3672.5
1 645      3593              3690
1 644      3660              3715
1 643      3630              3705
1 642      3665              3696
1 641    3657.5              3825
1 640      3830              3940
1 639      3920            4102.5
1 638      3935              4120
1 637  3922.727            4117.5
1 636  3832.667              4108
1 635  3776.667              3950
1 634 3793.3335              4000
1 633  3737.333              4000
1 632  3648.333              3925
1 631  3684.667              4130
1 630 4075.8335            4337.5
1 629 4248.3335              4425
1 628 4396.6665              4480
1 627  4453.333              4550
1 626 4384.1665              4525
1 625 4359.3335              4520
1 654      3166              3325
3 654      3899              3166
3 683      2550              3395
2 683      2550            3712.5
3 682      2550              3058
2 682      2550              3160
3 681      2550            2847.5
2 681      2550              2800
3 680      2550              2800
2 680      2550            2762.5
3 679      2550              2782
2 679      2550              2870
3 678      2550              2560
2 678      2550              2725
3 677      2550              2414
2 677      2550              2580
3 676      2550            2562.5
2 676      2550              2700
3 675      2550              2610
2 675      2550              2500
3 674      2550              2282
3 673      2550            1962.5
2 673      2550              2250
3 672      2550              1940
2 672      2550              2300
3 671      2550              1932
2 671      2550              2230
3 670      2550              2085
2 670      2550            2262.5
3 669  2751.667              2160
2 669  2751.667              2400
3 668  2842.857              2174
2 668  2842.857              2520
3 667 2841.4285 2251.666748046875
2 667 2841.4285              2600
3 666  2888.889              2260
2 666  2888.889              2670
3 665   2966.25              2395
2 665   2966.25            2837.5
3 664      3045           2453.75
2 664      3045              2875
end
format %tm ym

my code:
capture program drop xtby
program define xtby
xtcointtest pedroni mcprice2 mcprice3
matrix b = r(stats)
svmat b,names(c)
matrix c = r(p)
svmat c,names(d)
end

runby xtby,by(pair)


the error returns:

after this the whole data was disappear.

[Qingshi Wang]

sorry for the forgetting of using format of my codes

capture program drop xtby
program define xtby
xtcointtest pedroni mcprice2 mcprice3
matrix b = r(stats)
svmat b,names(c)
matrix c = r(p)
svmat c,names(d)
end

runby xtby,by(pair)

[Joro Kolev]

Seems to me that what you would like to do is not possible and does not make sense, because -xtcointtest- is a panel data cointegration test. That is, you need to have panel data to apply this procedure, you cannot apply the procedure panel by panel on individual panels.

[Qingshi Wang]

Seems to me that what you would like to do is not possible and does not make sense, because -xtcointtest- is a panel data cointegration test. That is, you need to have panel data to apply this procedure, you cannot apply the procedure panel by panel on individual panels.

Thanks for your reminding！
I try to use

 dfuller d.mcprice3
  dfuller d.mcprice2
  reg d.mcprice2 d.mcprice3
  predict e,residual
  dfuller e,lag(1)

to check whether the xtcointtest's estimation results is the same as the traditional way(just ADF test). I find it is actually not the same so may be I should quit to use xtcointtest to do the cointegration test for each individual in panel.

[Joro Kolev]

I think you should just use -xtcointtest- as it is, on the whole panel. From the help file:

xtcointtest performs the Kao (1999), Pedroni (1999, 2004), and Westerlund (2005) tests of cointegration on a panel dataset. Panel-specific means (fixed effects)
and panel-specific time trends may be included in the cointegrating regression model.

All tests have a common null hypothesis of no cointegration. The alternative hypothesis of the Kao tests and the Pedroni tests is that the variables are
cointegrated in all panels. In one version of the Westerlund test, the alternative hypothesis is that the variables are cointegrated in some of the panels. In
another version of the Westerlund test, the alternative hypothesis is that the variables are cointegrated in all the panels.

I see no reason for why you should try and reinvent the wheel by attempting to conduct the test panel by panel.

If you do the test panel by panel, you will have a severe multiple testing problem, and the p-values would be incorrect.

[Clyde Schechter]

I do not work with this kind of time-series data in my work, so I have no knowledge about the -xtcointtest- command. And I certainly defer to Joro Kolev on those aspects of this thread. I am responding, instead, as one of the authors of -runby- to point up why this program -xtby- is problematic for -runby- and suggests ways to fix it.

First, before attempting to use any program with -runby-, you should always test it on some single values. So before running -runby xtby, by(pair)-, try

preserve
xtset pair ym
keep if pair == 2
xtby
restore

And if this runs successfully, try it again with a few more. This will uncover most problems in program -xtby-. For example, when I try this with your example data I get an error message stating that the command cannot be used when there are time gaps in the data. So then you have to modify your code to something that will work with gaps, or fix the gaps in the data.

Once the program seems to reliably run with several selected individual values, there are a few extra considerations for using it with -runby-. I'll discuss those next--but these only come into play after you have a program that successfully runs as a standalone.

Remember that the data set is removed from memory by -runby-, converted into a Mata matrix, and by-group segments of the data are then fed into memory sequentially, acted upon by program -xtby-, and the results all appended, and finally the appended results are placed in memory. This raises a particular problem for survey data, survival data, panel data, or time-series data: a Mata matrix, unlike a data set, cannot be -svyset-, -stset-, -xtset-, nor -tsset-. So -runby- loses those settings. So, the program -xtby- must itself include an -xtset- command if it includes -xt- commands.

(Of course, some thought must be applied here. If the -by()- option of -runby- is the same as the panel variable in -xtset-, then under -runby- control, the data that -xtby- receives will no longer be panel data--it will be a simple time series, and the use of -xt- commands may no longer be appropriate. Similarly, taking a survey data set and running programs on subsets the way -runby- does will often produce incorrect results, just as using -if- conditions with -svy- commands often does.)

The other general tip for debugging code that calls -runby- is to, initially, use the -verbose- option. This will cause -runby- to report the output, including error messages, of -xtby-, so you can see what is going wrong directly. Depending on the level of detail needed, you might need to amplify this by beginning program -xtby- itself with -set trace on-, so that you get to see how macros are expanded. Again, this is best done on a relatively small subset of the data consisting of several -by()- groups, because the output of thousands or more iterations of -runby- can be overwhelming.

Once you can successfully run your program under -runby- on a few small subsets like that, you are ready to go to production mode. There you can remove the -verbose- option and run everything on the whole data set. Obviously, there is always the possibility that some subset(s) of the full data will produce errors that you did not pick up in your preliminary debugging, so you may have to revisit the earlier steps. But usually only one or two cycles of this development process are needed. And as you gain experience with -runby- you will be able to anticipate many of these problems in advance.

Finally, since people generally use -runby- with large data sets that are too time-consuming for processing with -levelsof- and -foreach { ... if ...-, I also recommend using the -status- option when running in production mode. It's one of my favorite features of -runby-: it gives periodic updates of how much progress has been made and an estimate of the time remaining to completion.

[Qingshi Wang]

I think you should just use -xtcointtest- as it is, on the whole panel. From the help file:



I see no reason for why you should try and reinvent the wheel by attempting to conduct the test panel by panel.

If you do the test panel by panel, you will have a severe multiple testing problem, and the p-values would be incorrect.

Thanks for your suggestion! I totally agree what you said if I do the test panel by panel I will not get a correct cointeration result for the whole panel. But I want to insist on doing the test for each individuals due to my research goal.

The individuals,'pair's, are city pairs and I want to see how the different city pairs cointegration are. If I use -xtcointtest- just for the whole panel I can only get one result and can't know the heterogeneity of the individuals,that is not satisfied my research goal. The panel here is just a storage form for me and what I am really concerned about is the time series subsets in the panel.

My original intention for using -xtcointtest- for time series data is because of the command's convenience:not needed for several command but one. After your reminding I find it give me a different result from traditional way(I do a test for one individual to check) so I will quit to use -xtcointtest-.( I still need -runby- or the other similar commands to run the data by group.)

[Joro Kolev]

Very good, Qingshi. In #4 you show that you know how to perform the Engle-Granger test of a single cointegrating relationship in a single time series data.

In #6 Clyde explained in great detail how to make his -runby- work.

I can also bring to your attention the user written command -egranger-
net describe egranger, from(http://fmwww.bc.edu/RePEc/bocode/e)
it does automatically what you do with a couple of commands in your #4.

Given what you want to do, just use -runby- either on your code in #4, or on the user written -egranger-.

[Qingshi Wang]

I do not work with this kind of time-series data in my work, so I have no knowledge about the -xtcointtest- command. And I certainly defer to Joro Kolev on those aspects of this thread. I am responding, instead, as one of the authors of -runby- to point up why this program -xtby- is problematic for -runby- and suggests ways to fix it.

First, before attempting to use any program with -runby-, you should always test it on some single values. So before running -runby xtby, by(pair)-, try

preserve
xtset pair ym
keep if pair == 2
xtby
restore

And if this runs successfully, try it again with a few more. This will uncover most problems in program -xtby-. For example, when I try this with your example data I get an error message stating that the command cannot be used when there are time gaps in the data. So then you have to modify your code to something that will work with gaps, or fix the gaps in the data.

Once the program seems to reliably run with several selected individual values, there are a few extra considerations for using it with -runby-. I'll discuss those next--but these only come into play after you have a program that successfully runs as a standalone.

Remember that the data set is removed from memory by -runby-, converted into a Mata matrix, and by-group segments of the data are then fed into memory sequentially, acted upon by program -xtby-, and the results all appended, and finally the appended results are placed in memory. This raises a particular problem for survey data, survival data, panel data, or time-series data: a Mata matrix, unlike a data set, cannot be -svyset-, -stset-, -xtset-, nor -tsset-. So -runby- loses those settings. So, the program -xtby- must itself include an -xtset- command if it includes -xt- commands.

(Of course, some thought must be applied here. If the -by()- option of -runby- is the same as the panel variable in -xtset-, then under -runby- control, the data that -xtby- receives will no longer be panel data--it will be a simple time series, and the use of -xt- commands may no longer be appropriate. Similarly, taking a survey data set and running programs on subsets the way -runby- does will often produce incorrect results, just as using -if- conditions with -svy- commands often does.)

The other general tip for debugging code that calls -runby- is to, initially, use the -verbose- option. This will cause -runby- to report the output, including error messages, of -xtby-, so you can see what is going wrong directly. Depending on the level of detail needed, you might need to amplify this by beginning program -xtby- itself with -set trace on-, so that you get to see how macros are expanded. Again, this is best done on a relatively small subset of the data consisting of several -by()- groups, because the output of thousands or more iterations of -runby- can be overwhelming.

Once you can successfully run your program under -runby- on a few small subsets like that, you are ready to go to production mode. There you can remove the -verbose- option and run everything on the whole data set. Obviously, there is always the possibility that some subset(s) of the full data will produce errors that you did not pick up in your preliminary debugging, so you may have to revisit the earlier steps. But usually only one or two cycles of this development process are needed. And as you gain experience with -runby- you will be able to anticipate many of these problems in advance.

Finally, since people generally use -runby- with large data sets that are too time-consuming for processing with -levelsof- and -foreach { ... if ...-, I also recommend using the -status- option when running in production mode. It's one of my favorite features of -runby-: it gives periodic updates of how much progress has been made and an estimate of the time remaining to completion.

Thanks dear Clyde for your valuable details for using -runby-!

I try to update -xtby- to adapt for -runby- with the help of your details.

I first use the option verbose, then I find two errors.
Firstly, I find it said 'the command does not allow gaps in data' (the same as what you tested). This error comes from -xtcointtest- and I chose to use -tsfill- to balance the panel.(-tsfill- just fill the time and individual values but fill the other variables as missing, that's enough for -xtcointtest- to work. I will try some interpolation methods to deal with the missing value next.)

Secondly, another error is that my command -xtby- need use -xtset-, however, after I used -xtset- to set the panel(outside -xtby-), the -xtby- still cannot run with -runby- for the same error. According to your description of how -runby- works, I think that is because -runby- actually creates new data subsets in memory sequentially so -xtset- should be defined before -xtcointtest- for each time series subsets.

After correction, the -xtby- could be:

 capture program drop xtby
 program define xtby
 xtset pair ym
 tsfill
  xtcointtest pedroni mcprice2 mcprice3
 matrix b = r(stats)
  svmat b,names(c)
   matrix c = r(p)
  svmat c,names(d)
  end

This could run successfully.

Your great details help me a lot to understand how -runby- works and taught me how to check the errors. As dear Joro suggested above, I will then rewrite -xtby- without using -xtcointtest- and after that I think I can combine -xtby- with -runby- successfully and correctly.

[Qingshi Wang]

Very good, Qingshi. In #4 you show that you know how to perform the Engle-Granger test of a single cointegrating relationship in a single time series data.

In #6 Clyde explained in great detail how to make his -runby- work.

I can also bring to your attention the user written command -egranger-
net describe egranger, from(http://fmwww.bc.edu/RePEc/bocode/e)
it does automatically what you do with a couple of commands in your #4.

Given what you want to do, just use -runby- either on your code in #4, or on the user written -egranger-.

Thanks dear Joro! -egranger- is helpful for me to do my research. I will use it with -runby- as you suggested.