xtset for intra-daily data (every half hour)

[River Huang]

Dear All, I have this (intra-daily, every 30 minutes) dataset for many companies (merchandise, but only one company here) and many days (daytime).

* Example generated by -dataex-. To install: ssc install dataex
clear
input int merchandise str25 daytime double(all f) float i double mj
1101 "2015-01-05 09:30:00+00:00"  .6322220282771862 .35140275955200195   .00921659    .6148881316184998
1101 "2015-01-05 10:00:00+00:00"  .6251472320376914 .21915708482265472    .0387931    .5218638181686401
1101 "2015-01-05 10:30:00+00:00" .48404255319148937  .3038821816444397   .02284264   .15426497161388397
1101 "2015-01-05 11:00:00+00:00"  .6306740027510316 .08415841311216354   .00967742    .5810810923576355
1101 "2015-01-05 11:30:00+00:00"  .5645933014354067 .10311750322580338  .010928961     .617977499961853
1101 "2015-01-05 12:00:00+00:00"  .7880671224362958  .4464172422885895           0   .37991267442703247
1101 "2015-01-05 12:30:00+00:00"  .8673693456302152  .2398567944765091   .05337078    .5466297268867493
1101 "2015-01-05 13:00:00+00:00"  .6915838996340826  .2481296807527542   .15284973   .19310344755649567
1101 "2015-01-05 13:30:00+00:00" .31804670912951166 .04122137278318405   .12878788   .11664074659347534
1101 "2015-01-06 09:30:00+00:00" .49465784430503845  .3459424376487732  .014405205    .5647687911987305
1101 "2015-01-06 10:00:00+00:00"  .5749144554138184 .19092218577861786  .011135858  .002812939463183284
1101 "2015-01-06 10:30:00+00:00"  .7315515875816345 .22560706734657288 .0027322404   .32138729095458984
1101 "2015-01-06 11:00:00+00:00" .40780141949653625   .093478262424469           0                    0
1101 "2015-01-06 11:30:00+00:00" .42289817333221436 .08409250527620316   .02898551                    0
1101 "2015-01-06 12:00:00+00:00"  .5283964276313782  .1515800803899765 .0024213076   .02286585420370102
1101 "2015-01-06 12:30:00+00:00"  .5427666306495667 .19020172953605652  .017605634   .43065693974494934
1101 "2015-01-06 13:00:00+00:00"  .5771617889404297 .12115505337715149  .005154639   .13809524476528168
1101 "2015-01-06 13:30:00+00:00" .35843345522880554 .04514441639184952   .11184573   .15654951333999634
1101 "2015-01-07 09:30:00+00:00"  .4490748345851898 .31891891891891894  .011627907    .5369960069656372
1101 "2015-01-07 10:00:00+00:00"  .5733739733695984 .36194690265486723  .011470986   -.3425774872303009
1101 "2015-01-07 10:30:00+00:00"  .4453725218772888 .15606936416184972  .011647254 .0019841270986944437
1101 "2015-01-07 11:00:00+00:00"  .6164215803146362 .30825688073394497  .009009009   -.3000663220882416
1101 "2015-01-07 11:30:00+00:00"  .5608547329902649 .14027149321266968  .015492958  -.13012729585170746
1101 "2015-01-07 12:00:00+00:00"  .5167531967163086 .24776119402985075  .032467533    .1510416716337204
1101 "2015-01-07 12:30:00+00:00"  .4741315543651581 .08921933085501858           0    .3311688303947449
1101 "2015-01-07 13:00:00+00:00"  .6261870265007019 .12844036697247707  .015873017    .5442193746566773
1101 "2015-01-07 13:30:00+00:00" .40022629499435425 .05528765279802624    .1766941    .3100692629814148
1101 "2015-01-08 09:30:00+00:00" .41960981488227844  .2408728450536728   .02857143   .32739904522895813
1101 "2015-01-08 10:00:00+00:00"  .5005064010620117   .306892067193985  .010387158   .20570151507854462
1101 "2015-01-08 10:30:00+00:00"  .4917241334915161 .12754766643047333  .029411765                    0
end

My question is: How can I -xset merchandise xxx- (how to deal with daytime variable) so that I can take lags of the variables (all f i mj)? Thanks a lot.

[Nick Cox]

30 min is 30 * 60000 ms, so 1.8 million ms. Hence following your data example I get

keep merchandise daytime
gen double ndaytime = clock(daytime, "YMD hms##")
format ndaytime %tc

xtset merchandise ndaytime, delta(18e5)
       panel variable:  merchandise (strongly balanced)
        time variable:  ndaytime, 1.736e+12 to 1.736e+12, but with gaps
                delta:  1800000 units

list

     +-----------------------------------------------------------+
     | mercha~e                     daytime             ndaytime |
     |-----------------------------------------------------------|
  1. |     1101   2015-01-05 09:30:00+00:00   05jan2015 09:30:00 |
  2. |     1101   2015-01-05 10:00:00+00:00   05jan2015 10:00:00 |
  3. |     1101   2015-01-05 10:30:00+00:00   05jan2015 10:30:00 |
  4. |     1101   2015-01-05 11:00:00+00:00   05jan2015 11:00:00 |
  5. |     1101   2015-01-05 11:30:00+00:00   05jan2015 11:30:00 |
     |-----------------------------------------------------------|
  6. |     1101   2015-01-05 12:00:00+00:00   05jan2015 12:00:00 |
  7. |     1101   2015-01-05 12:30:00+00:00   05jan2015 12:30:00 |
  8. |     1101   2015-01-05 13:00:00+00:00   05jan2015 13:00:00 |
  9. |     1101   2015-01-05 13:30:00+00:00   05jan2015 13:30:00 |
 10. |     1101   2015-01-06 09:30:00+00:00   06jan2015 09:30:00 |
     |-----------------------------------------------------------|
 11. |     1101   2015-01-06 10:00:00+00:00   06jan2015 10:00:00 |
 12. |     1101   2015-01-06 10:30:00+00:00   06jan2015 10:30:00 |
 13. |     1101   2015-01-06 11:00:00+00:00   06jan2015 11:00:00 |
 14. |     1101   2015-01-06 11:30:00+00:00   06jan2015 11:30:00 |
 15. |     1101   2015-01-06 12:00:00+00:00   06jan2015 12:00:00 |
     |-----------------------------------------------------------|
 16. |     1101   2015-01-06 12:30:00+00:00   06jan2015 12:30:00 |
 17. |     1101   2015-01-06 13:00:00+00:00   06jan2015 13:00:00 |
 18. |     1101   2015-01-06 13:30:00+00:00   06jan2015 13:30:00 |
 19. |     1101   2015-01-07 09:30:00+00:00   07jan2015 09:30:00 |
 20. |     1101   2015-01-07 10:00:00+00:00   07jan2015 10:00:00 |
     |-----------------------------------------------------------|
 21. |     1101   2015-01-07 10:30:00+00:00   07jan2015 10:30:00 |
 22. |     1101   2015-01-07 11:00:00+00:00   07jan2015 11:00:00 |
 23. |     1101   2015-01-07 11:30:00+00:00   07jan2015 11:30:00 |
 24. |     1101   2015-01-07 12:00:00+00:00   07jan2015 12:00:00 |
 25. |     1101   2015-01-07 12:30:00+00:00   07jan2015 12:30:00 |
     |-----------------------------------------------------------|
 26. |     1101   2015-01-07 13:00:00+00:00   07jan2015 13:00:00 |
 27. |     1101   2015-01-07 13:30:00+00:00   07jan2015 13:30:00 |
 28. |     1101   2015-01-08 09:30:00+00:00   08jan2015 09:30:00 |
 29. |     1101   2015-01-08 10:00:00+00:00   08jan2015 10:00:00 |
 30. |     1101   2015-01-08 10:30:00+00:00   08jan2015 10:30:00 |
     +-----------------------------------------------------------+

However, the gaps could still be a problem. You need information on why they occur. Perhaps someone does a paid job 09:30 to 13.30, then answers questions on Statalist. Business calendat? (I don't know anything about them.)

[River Huang]

Dear Nick, Many thanks for this useful suggestion. The data are from my students. They download (using python) the data for many stocks traded in Taiwan. For every day, they only record the stock data every half an hour. That is the reason (for gaps).

[Phil Bromiley]

The gaps seem to reflect periods when the stock exchange was closed - it looks like it closes at 1:30 and opens at 9:30. This is standard for exchanges at least in the US. Often you'll see that relations during normal trading do not hold for the non-trading breaks.

[River Huang]

Dear Phil, Indeed, the stock exchange opens at 09:00 and closes at 1:30 in Taiwan. The students used a python code to extract the information every 30 minutes.

[Nick Cox]

The implication is then that

lags 1, 2, 3, 4, 5, 6, 7, 8 will only ever refer to values earlier the same day.

lags 9, 10 and some up will only ever yield missing.

I suspect that you need a customised business calendar to make useful analyses but I've never used one.

[River Huang]

Dear Nick, Thanks for the further comments. I don't know yet. Once the students come back for this question, I will make sure I understand their demand.