Variances and Covariances in the panel data

[Emilia Kosinska]

Hi everyone,
I just started using STATA and I am wondering how can I compute covariances and variances for my variables at each date separately? I have to compute covariance of each portfolio return(45 portfolios in total) with cash-flow news variable at each t (877 months) and variance of my new variable (N_cf - N_dr) (cash-flow news minus discount rate news) also at each t (877 months) from my data set.
Any help from your side would be much appreciated!

[Clyde Schechter]

You don't show any example data, so I'll illustrate the code using one of StataCorp's demonstration data sets

webuse nlswork, clear

gen new_variable_variance = .
levelsof year, local(years)
foreach y of local years {
    corr tenure - ln_wage, cov
    matrix M`y' = r(C)
    summ wks_ue
    replace new_variable_variance = r(Var) if year == `y'
}

The variance of your new variable will be in the newly created variable new_variable_variance in the observations for the corresponding time period (here year instead of month), and you will have a covariance matrix for the other things for each time period, with a name that reflects the time period.

That said, with 877 time periods and 45 portfolios, the covariance matrices alone will constitute a corpus of 868,230 parameters (after taking into account that the covariance of A and B is the same as the covariance of B and A). How do you plan to work with that mass of data and gain any understanding from it? Presumably you plan to reduce that data to some kind of manageable summary with a small number of parameters? Depending on what you have in mind, it might be possible to calculate those summary statistics directly without creating all those covariance matrices.

[Emilia Kosinska]

Thank you very much for your answer.
I need the covariances and variances only as inputs to my Cash-flow and Discount-rate Betas, which I will then cluster for my analysis. Here is part of my data, with FFS1BM1 to FFS5BM5 and RISK1 to RISK20 being portfolio returns during given period (month). Beta is defined as covariance of a portfolio (e.g. RISK1) returns with N_cf (cash-flow news), divided by Variance of (N_cf-N_dr), which I generated as my new variable: unexpected market return. Will the code which you provided above also apply in case of this data set?
Thank you a lot for your time and help!

9

[Carlo Lazzaro]

Emilia:
as per FAQ, please share excerpt/example of your data via -dataex-, not via attached spreadsheets (that most of us would not open anyhow due to the risk of active contents). Thanks.

[Emilia Kosinska]

Thank you Carlo for your advice! I am new on Statalist, will keep that in mind for any future posts.
Here the excerpt from my data with variables relevant for my question :

* Example generated by -dataex-. To install: ssc install dataex
clear
input double(N_cf N_dr FFS1BM1 FFS2BM2 RISK1 RISK2)
           .            .      .      .            .            .
 -.033836261   .093003782  .0294  .0154   .098431561   .053369589
  .013806923 -.0099646031  .0097  .0325  -.006869177  -.016850875
 -.019542589   .021121964 -.1454 -.0605   .012795722  -.013255239
  .013234259   .014067435 -.0444  .0027   -.01041236    .03928717
 -.051268713  .0016765276 -.2188  -.115  -.072819467  -.092606913
-.0089180822     .1184492    .16  .0701   .097970797    .12342361
-.0057395027   .053309089  .0124  .0364    .10368659   .074322254
   .02217415   .065471212 -.0407 -.0032    .10714314   .095049749
 -.037917945  -.009027897 -.1022  .0134   .042307561   -.03310962
 -.044908979   -.15797531 -.2612 -.1554   -.23418919  -.074050369
   .11080698   -.21019277 -.0168 -.1367   -.12402502  -.094466441
 -.067242916    .10229204 -.0026 -.0391  .0030961047  .0025784174
  .021477012   .033381586 -.0276  .0759 -.0017166322   .038348348
  -.01519003   .039505819  .1643  .0314  .0012624555  -.028341948
  .024144205   .052717936  .2307  .0552   .063637325    .12163323
 -.033077695   .010790481 -.0682 -.0306   .022209448  -.027886186
  .033621379  -.039547806 -.0024 -.0151  -.023305476 -.0013875824
 -.080638894  -.085027566 -.1481 -.1451   -.13662271   -.10001987
end

[Clyde Schechter]

Your example data does not contain a time variable, so what you want cannot be done with it.

[Emilia Kosinska]

I forgot to include variable Date while creating my dataex file. Now the right excerpt:

* Example generated by -dataex-. To install: ssc install dataex
clear
input long Date double(R_Me EtR_Me N_cf N_dr FFS1BM1 FFS2BM2 RISK1 RISK2)
192812 -.0005809628            .            .            .      .      .            .            .
192901   .047326726  -.011840795  -.033836261   .093003782  .0294  .0154   .098431561   .053369589
192902 -.0051605997 -.0090029199   .013806923 -.0099646031  .0097  .0325  -.006869177  -.016850875
192903  -.012909596   -.01448897  -.019542589   .021121964 -.1454 -.0605   .012795722  -.013255239
192904   .012861431  -.014440263   .013234259   .014067435 -.0444  .0027   -.01041236    .03928717
192905  -.062034141  -.012441956  -.051268713  .0016765276 -.2188  -.115  -.072819467  -.092606913
192906   .089804462  -.019726657 -.0089180822     .1184492    .16  .0701   .097970797    .12342361
192907   .039766262 -.0078033242 -.0057395027   .053309089  .0124  .0364    .10368659   .074322254
192908    .07562698  -.012018383    .02217415   .065471212 -.0407 -.0032    .10714314   .095049749
192909  -.055435702 -.0084898597  -.037917945  -.009027897 -.1022  .0134   .042307561   -.03310962
192910   -.22402302  -.021138735  -.044908979   -.15797531 -.2612 -.1554   -.23418919  -.074050369
192911   -.13487817  -.035492384    .11080698   -.21019277 -.0168 -.1367   -.12402502  -.094466441
192912   .013495591  -.021553529  -.067242916    .10229204 -.0026 -.0391  .0030961047  .0025784174
193001   .050306074 -.0045525239   .021477012   .033381586 -.0276  .0759 -.0017166322   .038348348
193002   .022537928  -.001777861   -.01519003   .039505819  .1643  .0314  .0012624555  -.028341948
193003   .070728177 -.0061339641   .024144205   .052717936  .2307  .0552   .063637325    .12163323
193004  -.024022458 -.0017352437  -.033077695   .010790481 -.0682 -.0306   .022209448  -.027886186
193005  -.015993871  -.010067445   .033621379  -.039547806 -.0024 -.0151  -.023305476 -.0013875824
193006   -.17418238  -.008515923  -.080638894  -.085027566 -.1481 -.1451   -.13662271   -.10001987
end

[Clyde Schechter]

Thank you, but that variable you call Date is not a Stata date variable (unless you have somehow acquired data about events in the year 2,488), and in order to turn it into one, I need an explanation of how the numbers related to year month and day. I imagine the first two digits are the last two digits of the year, the third and fourth two digits are the day of the month, and the final two digits are the month of the year. But can you confirm that?

[Emilia Kosinska]

Thank you for your reply. I should have explained the Data variable in more detail. Taking the first row as an example (192812), the four first digits make a year (in this case 1928) and the last two digits are the month (12- December).

[Clyde Schechter]

OK. Your example data, actually, is still not sufficient to develop and test the code because you have only one observation for each month, so that no covariances within months can be calculated. But I have worked it out by expanding your example data and padding it with random number observations just to try out some code. Your situation is also made somewhat complicated by the occurrence of missing values which make the covariance calculations fail in some cases. Anyway, the following code does what you have asked for, taking that into account. The beta is created as a new variable in the data set. The covariance matrices are stored with names like M1930m2 for the February 1930 covariance matrix. You can -matrix list- them or use them in whatever way you like. When there is a month where the data are insufficient to allow calculations of the covariance matrix or the beta, you will get an error message to that effect in new variables comment1 and comment2. The same is true when a month has no usable data at all. If for any other reason the calculations fail, comment1 or comment2 will contain a message stating that an unexpected error has occurred. It behooves you, after running this code, to check these two comment variables: if there are any unexpected errors, you should investigate the data in those months to find out why. And if there are any months with no or insufficient observations for the calculations that should have enough data, then you need to check the data for those months as well to identify the problem and fix it.

//    CREATE A REAL STATA INTERNAL FORMAT MONTHLY DATE VARIABLE
gen year = floor(Date/100)
gen month = mod(Date, 100)
gen mdate = ym(year, month)
format mdate %tm
assert !missing(mdate)
drop year month

gen N_diff = N_cf - N_dr

capture program drop one_month
program define one_month
    local month: display %tm mdate[1]
    local month: subinstr local month " " "", all
    capture noisily corr FFS* RISK*, cov
    if c(rc) == 0 {
        matrix M`month' = r(C)
    }
    else if inlist(`c(rc)', 2000, 2001) {
        gen comment1 = "NO OR INSUFFICIENT OBSERVATIONS FOR COVARIANCE MATRIX"
    }
    else {    // UNEXPECTED ERROR
        gen comment1 = "UNEXPECTED ERROR CALCULATING COVARIANCE MATRIX"
    }
    capture noisily corr RISK1 N_cf, cov
    if c(rc) == 0 {
        local numerator = r(cov_12)
        summ N_diff
        local denominator = r(Var)
        gen beta = `numerator'/`denominator'
    }
    else if inlist(`c(rc)', 2000, 2001) {
        gen comment2 = "NO OR INSUFFICIENT OBSERVATIONS FOR BETA"
    }
    else {
        gen comment2 = "UNEXPECTED ERROR CALCULATING BETA"
    }
    exit
end

runby one_month, by(mdate) status

To run this code you need to install the -runby- command, written by Robert Picard and me, available from SSC.

In the future, when asking for help writing code, be sure to show example data, and please choose your example so that it is suitable for doing the calculations you want help coding.

[Emilia Kosinska]

Thank you a lot for the code! I run it and it looks like the last command
runby one_month, by(mdate) status
deletes my data completely, so that there are no variables left. Could you possibly explain why this might be the case? I am using STATA 13.0
Thank you for your help and time!

[Clyde Schechter]

When I run the code in #10 with the example data in #7 it runs with no difficulties and no data is deleted (although, because there is only one observation per monthly date, it produces no covariances or betas, just messages indicating that there aren't enough observations to do the calculations). Please post a new example of your data that exhibits the difficulty you are having so I can try to troubleshoot it.

[Emilia Kosinska]

Here more extensive example of my data. I have also the same data set with quaterly observations. (I attached the code below). As already mentioned there are 878 observations for monthly data and 294 for quaterly. There are 20 RISK portfolios (I attached only 5 in the example) and 25 FFS_BM_ portfolios, which are built in similar pattern as those attached (so the next group is FFS2BM1, FF2BM2, FFS2BM3, FFS2BM4, FFS2BM5 etc.until FFS5B5).
In general my task is to first replicate the methods used by John Campbell and Tuomo Vuolteenaho in their paper: "Bad Beta, Good Beta" and then cluster (replicated) betas. Since the authors published both monthly and quaterly data but used monthly for their calculations, I have also decided to use that data set. I attached the exact formulas for cash-flow and discount-rate betas that appear in the paper.
Many thanks!

* Example generated by -dataex-. To install: ssc install dataex
clear
input long Date double(R_Me TY PE VS EtR_Me N_dr N_cf Rrf FFS1BM1 FFS1BM2 FFS1BM3 FFS1BM4 FFS1BM5 RISK1 RISK2 RISK3 RISK4 RISK5)
192812  -.0005809628  -.63 3.2207675 1.8571141            .            .            .          .      .      .      .      .      .            .            .            .            .           .
192901    .047326726  -.75 3.2885226 1.8517692  -.011840795   .093003782  -.033836261 .004583356  .0294  .0725  .0359  .0342   .045   .098431561   .053369589   .032101969   .088620282   .11580332
192902  -.0051605997  -.85 3.2902403 1.8416018 -.0090029199 -.0099646031   .013806923 .004691455  .0097 -.1611 -.0396  .0139 -.0096  -.006869177  -.016850875    .01740372   .003980578  .019463738
192903   -.012909596  -.72 3.3042084 1.8296557   -.01448897   .021121964  -.019542589 .004899468 -.1454  .1695  .0471 -.0828 -.0147   .012795722  -.013255239   .032851456 -.0097782361 -.026224244
192904    .012861431  -.85 3.2948188 1.8084725  -.014440263   .014067435   .013234259 .005000773 -.0444 -.0688  .0905  .0544  -.019   -.01041236    .03928717  .0037951624    .01318865  .015484315
192905   -.062034141  -.81  3.306277 1.8335974  -.012441956  .0016765276  -.051268713 .004999944 -.2188 -.0568 -.1531 -.1618 -.1351  -.072819467  -.092606913   .032732422   -.10570624  -.13486302
192906    .089804462 -1.01 3.3217432 1.9047829  -.019726657     .1184492 -.0089180822 .004999859    .16  .0111  .0207  .0862  .0674   .097970797    .12342361   .050504804    .10156093   .15872836
192907    .039766262  -.79 3.4036583 1.8825641 -.0078033242   .053309089 -.0057395027 .004999651  .0124 -.0104 -.0369 -.0613  .0276    .10368659   .074322254   .039406053 -.0023114354  .038320629
192908     .07562698   -.7 3.4555554 1.8568828  -.012018383   .065471212    .02217415 .005108501 -.0407 -.0866 -.0779  .0283 -.0345    .10714314   .095049749   .083793891   .069354931   .10435429
192909   -.055435702  -.65  3.491234 1.8646432 -.0084898597  -.009027897  -.037917945    .005208 -.1022 -.0908 -.1219 -.0563 -.0938   .042307561   -.03310962  -.048294014  -.056207117 -.084960729
192910    -.22402302  -.76 3.3760604 1.8180527  -.021138735   -.15797531  -.044908979 .005208661 -.2612 -.2037 -.2292 -.2077 -.1851   -.23418919  -.074050369   -.12646046   -.23583928  -.27318742
192911    -.13487817  -.65 3.0651417 1.7906505  -.035492384   -.21019277    .11080698 .004483337 -.0168   -.12  -.098 -.1184 -.1559   -.12402502  -.094466441   -.11070353   -.15414495  -.16960838
192912    .013495591  -.14 3.0983275 1.7723598  -.021553529    .10229204  -.067242916 .004166324 -.0026 -.2456 -.1234 -.0911 -.0438  .0030961047  .0025784174   .027559616   .038185668  .060928607
193001    .050306074  -.21 3.1068586 1.7823571 -.0045525239   .033381586   .021477012 .004067247 -.0276   .179  .2242  .1107  .1209 -.0017166322   .038348348   .039177107   .053244606   .08175016
193002    .022537928  -.39 3.1618017 1.7658294  -.001777861   .039505819   -.01519003  .00385823  .1643  .0122  .0225 -.0328  .0203  .0012624555  -.028341948   .045348488   .050389769   .07232056
193003    .070728177  -.36  3.193036 1.7566879 -.0061339641   .052717936   .024144205 .003233572  .2307  .1422  .1666  .1154  .1063   .063637325    .12163323   .086609555   .084963461  .085272967
193004   -.024022458  -.16 3.2488439  1.746514 -.0017352437   .010790481  -.033077695 .003232954 -.0682  .0041 -.0616 -.0686 -.0565   .022209448  -.027886186  -.054229383  -.061597591 -.017558247
193005   -.015993871  -.16 3.1815688 1.7606415  -.010067445  -.039547806   .033621379 .003024639 -.0024 -.0326 -.0672 -.0659 -.0512  -.023305476 -.0013875824  .0087557813  -.032689813 -.019074207
193006    -.17418238  -.08 3.0693161 1.9953584  -.008515923  -.085027566  -.080638894 .002816595 -.1481 -.2399 -.2264 -.1291 -.1952   -.13662271   -.10001987   -.16875537   -.21355046    -.238902
193007    .037662938   .11 3.0420564 2.0477324   -.02423089    .10030766  -.038413831   .0026087  .0111  .1355  .2009  .0287  .0239    .01907
> 906   .056151854   .012388659   .035058941  .060214449
193008   .0010367072   .16 3.0235322 2.0431846 -.0033033035  -.026566241   .030906251 .002500162   -.09 -.1849 -.1185 -.0407 -.0399  -.019545
> 721 -.0075822756  -.037073199  -.060542743 -.003060672
193009    -.13521674   .11 3.0174617 2.0934404 -.0061123857  -.035573841  -.093530512 .002499565 -.2922  -.307 -.2846 -.2143 -.1958   -.11893
> 167   -.13591146   -.14586449   -.16204847  -.20345967
193010   -.089919813   .18 2.8638298 2.0866668  -.019818738  -.097258975     .0271579 .002500564 -.2913 -.1758 -.0674 -.1133 -.1009  -.073955
> 586  -.074445153   -.10418838   -.10839683  -.14736663
193011   -.028966747   .18 2.7829916 2.0993067  -.012797987  -.026998501   .010829741 .002399748  .1813  .0152 -.0248 -.0012 -.0572   -.03121
> 055  -.034989817  .0023708344  -.059776504 -.013840171
193012   -.082674461   .25 2.7124927 2.1785421 -.0060423085  -.051935829  -.024696323   .0023996 -.1985 -.1896 -.2695 -.1272 -.1767  -.081947
> 452   -.11073553   -.13139179  -.056929112  -.16316552
193101    .058128412   .32 2.7409704 2.1235241  -.010630567   .079554558  -.010795579 .002291752 -.2396   .278 -.0638  .1587   .207   .086873
> 448    .06042789    .03197303   .085173359   .12383086
193102     .10095341    .2 2.8131685 2.0822838  .0033687172   .070956607    .02662809 .002083567   .222  .3136  .3869  .2454  .2535   .030457
> 082    .05792388    .16562503    .11698192   .34594699
193103   -.068466598   .53 2.8308013 2.1160557  .0061117844  -.048425414  -.026152969 .002083206 -.0801  -.043 -.1188 -.0511 -.1049  -.085861
> 023   -.12259706   -.13479744  -.056557848  -.10036053
193104    -.10554394   .56 2.7293183 2.1804952 -.0084267628  -.074149646  -.022967528 .001983768 -.1518 -.2626 -.0883 -.1698  -.164  -.082707
> 071   -.12701784   -.14723471   -.14476577  -.19876025
193105    -.14427058   .61 2.6265056 2.2173272   -.01107085  -.081015312   -.05218442 .001774975 -.0695  .1183 -.2685 -.1086  -.156   -.12132
> 651   -.18333241   -.20805491   -.18000125  -.18841683
193106     .12726414  1.03 2.5925127 2.1440688  -.013378329   .070432422    .07021005 .001666815  .2065  .0812  .1596  .1217  .1732    .15986
> 515    .21052249      .230295    .21676533   .36425094
193107   -.067089267  1.05 2.6237757 2.1290568   .016315248  -.066100698  -.017303816 .001666407 -.1841 -.1141  -.183 -.0579 -.0446  -.053903
> 167  -.073793241   -.12816152    -.0897646  -.14435557
193108 -.00083573872  1.09 2.5919472  2.169208 -.0021443864   .015922061  -.014613413 .001666782 -.0048 -.0025 -.0144 -.0342 -.0614   .022478
> 444   .044442787 -.0035934075   .019265373    .0414717
193109    -.34416937  1.45 2.4293368  2.255968   .004315705    -.2592341  -.089250974 .001666366 -.4936 -.3051 -.2877 -.3413 -.3089   -.25779
> 702    -.3193598   -.32233491    -.4148504  -.38878948
193110    .074392095   .73 2.2846174 2.2523891  -.024488045   .075903649   .022976491 .003333461  .1217 -.0296  .0781  .1356  .0563   .092364
> 571    .12464762   .068705651    .18230368   .08682088
193111   -.096758816   .18 2.2968269 2.2705994   .012502128  -.052949384   -.05631156 .003233268 -.2383 -.2192 -.0775 -.1296 -.0852  -.077389
> 286   -.12330053  -.088504956    -.1133338  -.27457707
193112    -.14767644   .13 2.0864168 2.3758314 -.0075564953   -.16234286   .022222913 .003233757 -.2537 -.0254 -.2693  -.251 -.2467   -.16474
> 441   -.16800117   -.20162833   -.22946828  -.26101713
193201   -.013803167  -.06 2.0671479 2.2988053  -.010954621   .033895761  -.036744307 .003232654  .0622 -.0693  .0783  .1133  .0697 -.0071635
> 441  .0053276503   .060847788   .041582499  .055419232
193202    .052612803  -.27 2.0561428 2.3155941  .0016922521   .028471475   .022449076 .003233851 -.1285  .0027   .059  -.034  .0405   .073931
> 951   .044209486  .0078468746  -.025729428   .16317563
193203    -.11873446  -.23 2.0572521 2.3070669  .0065596766  -.070316461  -.054977675 .003025098 -.2306 -.0381  -.093 -.0937 -.1158  -.033221
> 917   -.11641787   -.13599329   -.14198936  -.23641191
193204     -.2012149  -.12 1.7806746 2.2988752 -.0092487301   -.26467233   .072706156 .002816704 -.4077  -.174 -.2134 -.2609 -.1709    -.1807
> 828   -.10785109   -.22109171   -.27379849  -.28058331
193205    -.23419034  1.36 1.6473528 2.3297257  -.012226678   -.11869572   -.10326794 .002399864 -.1294 -.3576  .0021 -.0488 -.0697    -.1126
> 135   -.23125764   -.25170051   -.26538295  -.35378836
193206   -.011461755  1.02 1.5006242 2.4491561 -.0043286592  .0093883908  -.016521487 .002191292  .3581 -.1698 -.0547 -.0538 -.0158   .047620
> 726   .086796574   .057980764   .095107652   .11237758
193207     .28371406  1.16 1.5472099 2.2453256   .015121063   .092358703     .1762343   .0020834  1.475  .0331  .2655  .4949  .5786    .33055
> 764    .39909914    .38615879    .50758003   .79063365
193208     .30923449   .65 1.9521714 2.1184226   .045720724    .29956428  -.036050514 .001774982  .4949 1.5117  .8451  .7769  .9099    .20638
> 553    .26179358    .35015488    .65980071   .79070329
193209   -.033263974   .61 2.0422095 2.2250925   .040604355  -.060117722  -.013750607  .00177488 -.2703 -.0662  .0466 -.0734 -.1722  -.062250
> 519  -.045741656  -.039384922   -.04045643 -.052937449
end

* Example generated by -dataex-. To install: ssc install dataex
clear
input long Date double(R_Me TY PE VS EtR_Me N_dr N_cf Rrf FFS1BM1 FFS1BM2 FFS1BM3 FFS1BM4 FFS1BM5 RISK1 RISK2 RISK3 RISK4 RISK5)
192812    .12475028  -.63 3.2207675 1.8571141             .            .            .             .            .           .           .           .            .           .            .             .           .            .
192903   .029256531  -.72 3.3042084 1.8296557   -.038889224    .10785189  -.039706131    .014241329     -.106354  .080632828  .043384034 -.034735077   .020690477   .10436306   .023249643    .082378998  .082817097    .10902987
192906   .040631752 -1.01 3.3217432 1.9047829   -.038528673   .087829828 -.0086694034    .015075707   -.10352382  -.11447486 -.042051542 -.021380674  -.086778177  .014665017   .070001628     .08705208 .0089354216   .039139246
192909    .05995754  -.65  3.491234 1.8646432   -.047649488    .19629581   -.08868878    .015394469   -.13047612  -.18770293  -.23653135 -.089298759   -.10070304   .25333763    .13627692    .074879436  .010797093   .057632931
192912    -.3454056  -.14 3.0983275 1.7723598   -.049942325    -.3845253   .089062024    .013922151   -.28054889  -.56826115   -.4499719  -.41665723    -.3843622  -.35483874   -.16587278     -.2095297  -.35158381   -.38167334
193003    .14357217  -.36  3.193036 1.7566879   .0031494299    .13967837 .00074437843     .01120042    .36766992   .33369376   .41376153   .19335355    .24767466  .063182837    .13164092     .17122614   .18871274    .23953424
193006   -.21419871  -.08 3.0693161 1.9953584   -.030147595  -.046793046   -.13725807   .0091016192   -.21859381  -.26833296   -.3548671  -.26338121    -.3026609  -.13772241   -.12926397    -.21415697  -.30761642    -.2754436
193009  -.096517094   .11 3.0174617 2.0934404   -.010058928  .0072921574  -.093750324   .0076277354   -.37087942  -.35649811   -.2026719  -.22628594   -.21177819   -.1194015  -.087412144    -.17051761  -.18751216   -.14642401
193012   -.20156103   .25 2.7124927 2.1785421   -.014371335   -.24818221   .060992517   .0073176859   -.30880873  -.34992217  -.36143485    -.241553   -.33446302  -.18700434    -.2200233    -.23307802  -.22494195   -.32408892
193103   .090615231   .53 2.8308013 2.1160557   .0055882806    .12294064   -.03791369   .0064724247  -.098217388   .54921705   .20366961   .35318276    .35564113  .031395202 -.0043554296     .06258715   .14558268    .36937381
193106   -.12255038  1.03 2.5925127 2.1440688  -.0087446753   -.22152976    .10772406   .0054353501  -.015151266  -.06352808  -.19753199  -.15685419   -.14709796 -.044394096   -.10024776     -.1255058  -.10844453  -.023960581
193109   -.41209437  1.45 2.4293368  2.255968    .021499987   -.16074683   -.27284753   .0050078915    -.6815588  -.42134141   -.4845058  -.43306593   -.41454527  -.28915282   -.34864921    -.45366055  -.48507438   -.49133298
193112   -.17004316   .13 2.0864168 2.3758314    .047297748   -.27432431   .056983402    .009832534   -.37029346  -.27407194  -.26876198  -.24519036   -.27557655  -.14986516   -.16680979     -.2214484   -.1608628   -.44852102
193203  -.079924823  -.23 2.0572521 2.3070669    .026209667  -.040253179  -.065881311   .0095216504   -.29682685   -.1046765  .044218465  -.01451279 -.0056994158  .033519014  -.066936055   -.067387274  -.12616949  -.018243618
193206     -.446867  1.02 1.5006242 2.4491561     .01991643   -.53045509   .063671662   .0074260655    -.1803939  -.69987618   -.2658889  -.36320613   -.25624289  -.24571161   -.25235686    -.41452877  -.44385929    -.5217132
193209    .55968458   .61 2.0422095 2.2250925    .082018018    .48605745  -.008390892   .0056438147    1.7015554   1.4780774   1.1599623   1.2013086    1.3189924   .47504021     .6158931     .69799037   1.1297997    1.5338106
193212   -.15655068    .9 1.8478489 2.4710202   .0040287148   -.16719928  .0066198829   .0040803828   -.28850254  -.48295364  -.51891397  -.53625732   -.46590054 -.077603176   -.10139478    -.17453076  -.29386305   -.31707629
193303   -.12119363   .12 1.7639007 2.3626043    .046322408    -.1194005  -.048115528   .0050160299  -.049518025  -.21128529  -.28613988  -.01728999   .041272825  -.14937511  -.058495154   -.077600031 -.028315325   -.20526428
193306    .63600875   .72 2.2819462  2.298467    .040496984    .51014389   .085367877   .0050165832    1.5419082   2.0771286   1.7781512   1.8953966     1.733572    .6881956    .81859817     1.0749526   1.0548967    1.3287004
193309   -.10052403   .94 2.3066893 2.4460944   -.013332628   .021758681   -.10895009   .0035463455   -.19894616  -.22897036  -.15395102  -.10938244   -.25950278  .011673153  -.051708805    -.13180946   -.1375328   -.13055162
193312   .025922995   .46 2.2560423 2.5477196     .02158164 -.0043295708  .0086709256   .0033368271   -.48316368   -.1677332  -.18069419  .095366198   -.14460339 -.015566804   .026179523 -.00080337097  .031478481  -.030610817
193403   .099633907   .67 2.3434459 2.3892259   .0058318245   .050372467   .043429615   .0032454132    .93968377   .58809759   .55605409   .48262874    .52535539  .078129583    .12979138     .21564419   .24842462    .18973417
193406  -.068838249  1.28 2.2792184 2.4696392   .0071516059  -.052725933  -.023263923   .0024017249   -.33254912  -.12430857 -.051795063  -.10462735   -.11049556  .023632931    -.1245006    -.13272673  -.17032888   -.17807953
193409  -.062210859  1.14 2.1798095 2.5468249    .025870258  -.077021019  -.011060098   .0022014815   -.05526265  -.21115651  -.21172976  -.15520224   -.14818917 -.028956867   -.12448995     -.1914268  -.17568195   -.17081091
193412   .063450254  1.18 2.2344085 2.6517267    .025303776   .084583746  -.046437267    .002201811   .074613988   .15076708   .18094449   .20943041   .079067713  .093470587    .12903485    .042690693  .040389879 -.0025792253
193503  -.090574809  1.69 2.1472461 2.7133913    .012453713  -.081505057  -.021523465  .00089380889   -.16992208  -.39199147  -.17091326  -.28286559   -.21277058 -.077221391   -.13887896     -.2278849  -.29066952   -.32900307
193506    .17308178  1.79 2.3367592 2.6528761    .030721039    .16944898  -.027088246  .00034133036      .205144 -.043301315   .21204829   .23976078    .12239697   .13265591    .22466701     .23666951   .16296174    .26106772
193509    .12079417  1.61  2.477612 2.6280946    .011266347    .12537375  -.015845928  .00039558633   .070331756   .50669035   .41069757   .31507631    .32259239  .058117387     .2175871     .16314083   .25911247    .28539343
193512    .16250934   1.6 2.6008892 2.5856493   .0048837235    .11528904   .042336577  .00031095385    .53421831   .17949996   .42281388   .43018879    .46720223   .12800445    .24617738     .21107364   .32429809    .31263112
193603   .097543911  1.68 2.7440192 2.4694844  -.0020784427     .1067279 -.0071055459  .00025428522    .45919944   .29543703   .36863184   .26220128    .43747001  .029637851    .08788889     .18915103   .27713285    .20519867
193606 -.0069662492  1.62 2.7438255 2.5319316 -.00012527291   .019683839  -.026524815  .00041507121   -.27732658  -.24726839  -.10614178 -.080117685   -.18880428  .026965953 -.0055344251   -.038213638 -.048831726  -.023537832
193609   .086598288  1.59 2.8416706 2.5221143   .0019125201    .10649831  -.021812545  .00047070669   -.05319435  .088620894   .20918429   .24449399    .17339032   .12523021    .11633828     .14893978    .1363108    .15618156
193612    .10174364  1.54 2.9102037 2.4377632   -.008068044   .059310442   .050501244  .00041475863    .22012015   .10693256   .18689049   .26549011    .21962621  .078192717    .16626159     .10850921  .098384819   .092632877
193703   .039351159  1.27 2.9739517 2.3271827  -.0098348532   .044275108  .0049109042   .0012054395 -.0032482198 -.017044906   .14981228    .1858812    .28024566  .039426212   .077854188     .19118205    .1462972    .16403617
193706   -.12901254  1.28 2.8314034 2.4930039  -.0098112235  -.082785434  -.036415886   .0015806469   -.37835636  -.22424524   -.3562444  -.22998008   -.37657713  -.11185068   -.14589709    -.15994956  -.22082012   -.23231111
193709     -.111366  1.33 2.7481621 2.5349238 -.00012041349  -.054406455  -.056839128   .0012218437   -.15689109  -.23859545  -.16647334  -.21435005   -.16832524  -.10911185   -.11739128    -.18467275  -.19092673   -.07566993
193712   -.22799573  1.57 2.4829122 2.5117296   .0023602342   -.25974179   .029385828  .00034206732   -.34779757  -.41883519  -.44604289  -.37703892   -.37633483  -.21832296    -.2328189    -.30693802  -.29653763   -.27115684
193803   -.20557469  1.72 2.4172208 2.5509157    .026779483   -.06242012   -.16993405  .00021401526   -.18248701  -.21361298  -.18258644  -.24947606   -.27595785  -.20711283   -.17951596    -.21377983  -.29802665   -.27776103
193806     .3074818  1.95   2.41311 2.5055259    .029830735    .01381468    .26383638 .000057691025    .58981412   .51267417    .5668762   .44178964    .42286987   .29680753    .38228149     .46751463   .51862673    .64212453
193809   .050639224  1.82 2.5643184 2.5458843   .0087369519    .13556581  -.093663541  .00011760445   .071553339 -.055866371   .17229561  .083641251  -.032203709 .0025825201   .024841018    .015275459    .1006358   .065560427
193812   .095621897  1.96 2.6572545 2.5705523    .010741464   .098043267  -.013162834 .000040830425   .052739778   .21871105   .16761924  .089388894   .053481624  .056691637     .1278743     .13696414   .13401058    .26053685
193903   -.15421872  2.03 2.6536644 2.6162633   .0049223778 -.0083065762   -.15083452 .000018280065   -.36330703  -.29726583  -.28086936  -.25751594   -.23568047  -.12350432   -.23582774    -.28860392  -.31245774   -.37660961
193906  .0092952309  1.86 2.5919211 2.4885233    .017475855  -.081891004   .073710379 .000023320164  -.007995497 -.032812983 -.016190743 -.017805353   -.04728126  .031622959  -.039183262   -.042528495    -.057109 -.0090714562
193909    .17588746  1.79 2.7205596 2.3860514    .014398397    .10558706   .055902004  .00013510452    .57815751   .54370251   .71600191    .5027206     .9146629  .045222186    .28945304     .45190724   .44486768    .61583731
193912  -.011363188  2.02 2.7072421  2.551046   .0020447579   .023291571  -.036699517 .000028460222   -.10960015  -.20460634  -.18071512  -.16005008   -.32780796  .018779601  -.024038765     -.0840026  -.10476488   -.15004633
end

[Emilia Kosinska]

Forgot to add that the betas should be calculated for two separate subsamples 1928:12-1963:6 and 1963:7-2001:12.

[Clyde Schechter]

Using your next example data, I still cannot reproduce your problem. When I run my code with it, the data set is preserved. No covariance matrices or betas are calculated because once again you show only one observation per month, so it is not possible to do within-month covariances. Indeed, if in your real data there is only one observation per month, then it is simply impossible in principle to do the calculations you are describing. Either you have gathered an inappropriate data set or you have misunderstood what the calculations you need are.

The reference you provide is not helpful: perhaps everybody in your field knows about this article, but those of us who are in other disciplines (many of us on the Forum are) have no idea where to find this. If you provide a reference, always provide a complete reference or, better still if possible, a link to a website where the article is available without charge.

I have not looked at your attachment. As is made clear in the Forum FAQ, which everyone is asked to read before starting a thread, attachments are discouraged here. I am among those who will not download attachments from strangers--particularly not Word documents, which can contain active malware.

Finally, "Forgot to add that the betas should be calculated for two separate subsamples 1928:12-1963:6 and 1963:7-2001:12." makes no sense to me. You are calculating a beta for every month, so for the purposes of this calculation there is no partition into subsamples of this nature.

My instinct based on all of the above is that your initial problem description is incorrect and has lead me to code a solution to the problem you described, but not to the problem you want to solve.