Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • VAR and Impulse response function

    Dear Statalists,

    I am working on a project where I need to
    1. estimate the reduced form VAR to predict the residuals ( the shock/ the innovation) and the contemporaneous terms ( details in picture)
    2. estimate the long-run (t=15) cumulative return responses to unit shocks of the innovations in structured VAR ( theta_rm, theta_x, theta_r). The short-run restriction is that rm can affect x and r, x can affect r, while r cannot affect rm and x contemporaneously.
    This is the reduced VAR This is the structured VAR This is the relationship of innovation between reduced and structured VAR, considering the contemporaneous terms.
    1.JPG 2.JPG3.JPG
    Code:
    var vwretd sdvol RET, lags(1/5)
    predict resid_rm, residuals equation (vwretd) 
    predict resid_x, residuals equation (sdvol) 
    predict resid_r, residuals equation (RET)
    
    reg resid_x resid_rm
    rename b_resid_rm b_1_0
    reg resid_r resid_rm resid_x
    rename b_resid_rm c_1_0
    rename b_resid_x c_2_0
    After estimating all the parameters in reduced VAR, I need to get the cumulative impulse response function in structure VAR.
    Code:
    gen sdvol_1 = sdvol - b_1_0*resid_rm
    gen r_1 = RET - c_1_0*resid_rm - c_2_0*resid_x 
    var vwretd sdvol_1 r_1, lags(1/5)
    irf create myirf, set(myirfs, replace) step(15)
    irf table cirf, impulse(vwretd) response(vwretd sdvol_1 r_1)  individual step(15)
    irf table cirf, impulse(sdvol_1) response(vwretd sdvol_1 r_1) individual step(15)
    irf table cirf, impulse(r_1) response(vwretd sdvol_1 r_1) individual step(15)
    Here I have the problem:
    1. What I am doing is basically deduct the contemporaneous terms in the variables of reduced form, and estimate the VAR all over again. Is it correct to do such procedure? Or is there other way to acquire the cirf at step =15 of structured VAR?
    2. In the irf file, with 3 variables, there are in total 3^2 impulse-response relationships. Yet, I am asked to calculate the response to unit shock of each innovation. Does this mean the theta is a actually a vector? E.g. the response to unit shock of vwretd is the sum of response (vwretd sdvol_1 r_1)?
    3. I noticed there is also -svar- in Stata. Is the mechanism of -svar- the same as my procedure from reduced VAR to structured VAR?
    4. How can I return the response values in each step stored in irf file? I have checked the reference: chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.stata.com/manuals/tsirfcreate.pdf
    With the reference, I tried
    Code:
    irf describe myirf
    mat list _dta(step)
    or 
    mat list r(step)
    But get the error
    Code:
    matrix _dta(step) not found
    matrix r(step) not found
    Sorry the questions are a bit messily presented. The main questions I would love to ask are the problem 1-4.

  • #2
    To elaborate the 4th question:
    My purpose is to return the values in irf table and save them in the original data.
    Now I can think of two ways to do so:
    1. To directly return the values and then generate new variables to store them
    Code:
    irf describe myirf
    mat list _dta(step)
    or 
    mat list r(step)
    But Stata shows the error
    Code:
    matrix _dta(step) not found
    matrix r(step) not found
    2. I am using -runby- to estimate VAR model through each groupd id.
    Code:
    egen id = group(PERMNO year)
    For each run of the program, I would love to save the irf file with the id stored in the file. This can help me map back to the original data.
    Code:
    program irf
    gen mydate= _n
    tsset mydate
    var vwretd sdvol_1 r_1, lags(1/5)
    irf create myirf, set(myirfs, replace) step(15)
    save mydata.dta,replace
    use "myirfs.irf",clear
    egen cirf_rm = total(cirf) if step ==15 & response == “vwretd”
    egen cirf_x = total(cirf) if step ==15 & response == “sdvol_1”
    egen cirf_r = total(cirf) if step ==15 & response == “r_1”
    gen id = `id'
    drop mydate
    duplicated drop id, force
    save myirfs.irf, replace
    use mydata.dta, clear
    merge m:1 id using myirfs
    end 
    
    use sample.dta, clear
    runby irf, by(id) verbose
    Somehow, the error
    Code:
    variable mydate already defined
    I am sure there is no variable named mydate before I activate -runby-, and I also drop mydate in the program after use.

    The data
    Code:
    * Example generated by -dataex-. For more info, type help dataex
    clear
    input double PERMNO long date double(RET vwretd) float(size sdvol year id resid_rm resid_x resid_r) double(b_1_0 c_1_0 c_2_0) float(sdvol_1 r_1)
    10006   3    .03865979239344597           4.648954e-06 71.43175  115862.5 1960 78             .          .            . 3854004.0653224494 .3777748265532016 1.8997299301524146e-07          .             .
    10006   4   .022332506254315376             .007798498   73.027    108150 1960 78             .          .            . 3854004.0653224494 .3777748265532016 1.8997299301524146e-07          .             .
    10006   6  -.014563106931746006   -.006302413000000001  71.9635    -50750 1960 78             .          .            . 3854004.0653224494 .3777748265532016 1.8997299301524146e-07          .             .
    10006   7  -.007389162667095661            -.003181282 71.43175    -30225 1960 78             .          .            . 3854004.0653224494 .3777748265532016 1.8997299301524146e-07          .             .
    10006  10  -.017369726672768593             -.01142776   70.191    -94050 1960 78             .          .            . 3854004.0653224494 .3777748265532016 1.8997299301524146e-07          .             .
    10006  11  -.005050505045801401            -.005640277  69.8365    -44325 1960 78  -.0016438435 -15300.732   .005418145 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -37989.62  -.0015227764
    10006  12  -.005076142027974129            -.005100299   69.482    -34300 1960 78   -.005836897 -18781.736 -.0023219332 3854004.0653224494 .3777748265532016 1.8997299301524146e-07 -11804.577   .0006969132
    10006  13   .030612245202064514    .005762501000000001   71.609     35350 1960 78     .00788807   59467.12    .03726222 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   4949.345    .016335184
    10006  14   .009900989942252636  .00008260490999999999   72.318    122400 1960 78  .00004419835  132563.67   .012045564 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  122229.66   -.015299224
    10006  17 -.0049019609577953815             -.00813024  71.9635    -50750 1960 78   -.006047334  -40619.66 -.0009043231 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -27443.55    .005099207
    10006  19  -.017241379246115685            -.002773081 70.72275    -29925 1960 78 -.00024211688 -28954.613  -.012614672 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -28991.88    -.01164932
    10006  20   .002506265649572015             .002360102     70.9     85000 1960 78    .000295396  77677.875  .0003378637 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   83861.54   -.012362026
    10006  21   .009999999776482582             .003016103   71.609     25250 1960 78    .002759174   29631.35   .012220737 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  14616.132   .0033284975
    10006  24  -.002475247485563159             -.01026663 71.43175  -15112.5 1960 78   -.010197697  -9707.642   .003749972 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  24189.467   .0032213756
    10006  25  -.012406948022544384             .001584803  70.5455    -44775 1960 78    .004685143  -23195.37  -.006358744 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -62831.56   -.009770383
    10006  26   -.01005025114864111            -.002537099  69.8365    -98500 1960 78   -.004734828 -101725.45   -.01075333 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -80251.95    .011063535
    10006  27   .007614213041961193   -.009771914000000001 70.36825     39700 1960 78   -.008005814   41638.09   .010966547 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   70554.44    .002728495
    10006  28  -.017632242292165756            -.009597647  69.1275   -117000 1960 78   -.007571897  -96577.06  -.012595053 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -87817.88   .0035752635
    10006  31  -.012820512987673283   .0051717510000000005 68.24125    -77000 1960 78    .007880263  -43454.12  -.002484098 3854004.0653224494 .3777748265532016 1.8997299301524146e-07 -107370.56   -.007542368
    10006  32     .0181818176060915    .015570800000000001   69.482     58800 1960 78     .01154266   50446.09    .01051406 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  14314.537   .0042378954
    10006  33  -.010204081423580647            -.008165478   68.773   -101850 1960 78    -.00983223 -103477.93  -.014654355 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -63956.54      .0131683
    10006  34   .007731958758085966           -.0005813497 69.30475     39100 1960 78    .004197796  65609.055    .01561624 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   22921.68   -.006317812
    10006  35 -.0025575447361916304            -.004769153  69.1275    -39000 1960 78   -.007022112  -41225.09   -.00510981 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -11936.75    .007926886
    10006  38 -.0025641026441007853   -.009837535000000001 68.95025    -87525 1960 78   -.007089827  -75647.89  .0018476216 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -60200.78    .014485312
    10006  39    .02570694126188755    .008982898000000001 70.72275   54862.5 1960 78    .009294443   63552.51    .02425308 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   19041.68    .010122474
    10006  40   .002506265649572015   -.005426065000000001     70.9     20000 1960 78   -.006415536   27971.71   .002525391 3854004.0653224494 .3777748265532016 1.8997299301524146e-07    44725.5  -.0003839756
    10006  41  .0024999999441206455  -.0048993000000000005 71.07725   35087.5 1960 78 -.00024883615   48431.46   .009528263 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  36046.516   -.006606666
    10006  42   .014962593093514442    .004666615000000001 72.14075   45787.5 1960 78    .004058123   40161.59   .015205628 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  30147.477    .005799919
    10006  45  .0024570024106651545   -.004848740000000001   72.318     76500 1960 78   -.004472319   79367.45   .005780797 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   93736.34    -.01093114
    10006  46  -.007352941203862429   -.008772124000000001 71.78625  -75937.5 1960 78   -.007330962  -71429.84  -.005370385 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -47683.95    .008986252
    10006  47   .007407407276332378             .006324385   72.318    137700 1960 78    .008513372  148543.92   .014437018 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  104889.43   -.024028065
    10006  48   .009803921915590763              .01368883   73.027    154500 1960 78     .01063673  142266.66   .009713168 3854004.0653224494 .3777748265532016 1.8997299301524146e-07     113506    -.02124119
    10006  49   .004854368977248669    .007340360000000001  73.3815    155250 1960 78    .005409687   142116.7    .00554966 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  134401.05    -.02418761
    10006  53   -.01690821163356304   -.004926641000000001  71.2545    -35175 1960 78   -.007512591  -61014.99   -.01722398 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -6221.444   -.002478944
    10006  55  -.002487562131136656             .004638512 71.07725  -55137.5 1960 78      .0054325  -51916.23  .0010344604 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -76074.38    .005322858
    10006  56   .007481296546757221    .004323939000000001   71.609     85850 1960 78    .001297356   73438.37    .00653275 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   80849.98   -.006960118
    10006  59  -.002475247485563159            -.000621328 71.43175  -35262.5 1960 78  -.0008026284  -40408.89 -.0003918516 3854004.0653224494 .3777748265532016 1.8997299301524146e-07 -32169.166    .005504563
    10006  60   .004962779115885496            -.001338409 71.78625     10125 1960 78  -.0009843444   8421.577    .00933375 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  13918.667   .0037347674
    10006  61  -.009876543655991554   -.006937054000000001 71.07725  -75187.5 1960 78    -.00741321  -78929.28  -.009785145 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -46616.96    .007918412
    10006  62  -.017456358298659325             -.01532618  69.8365   -113275 1960 78    -.01332579  -97358.66   -.01500417 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -61917.35    .006073305
    10006  63  -.017766498029232025  -.0037686520000000004 68.59575 -111262.5 1960 78   -.001343145  -96445.02   -.01227303 3854004.0653224494 .3777748265532016 1.8997299301524146e-07 -106086.02   .0010628566
    10006  66  -.018087854608893394             -.01105556   67.355    -42750 1960 78   -.012004664  -24400.89  -.017124373 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  3516.0244   -.008917284
    10006  67  -.010526316240429878   -.009165752000000001   66.646    -28200 1960 78   -.006401291  -2767.088 -.0041327123 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -3529.398   -.007582398
    10006  68   .005319148767739534    .011078020000000001  67.0005     23625 1960 78    .011663036   52196.46   .011477076 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -21324.39    -.00900277
    10006  69   .010582010261714458  -.0032073690000000003  67.7095     38200 1960 78   -.006247851   50093.53   .011008768 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   62279.24    .003425873
    10006  70  -.007853402756154537   .0074618060000000005 67.17775  -52112.5 1960 78    .011041815 -34750.063 -.0040470064 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   -94667.7   -.005423149
    10006  73   .013192611746490002             .001969536   68.064     24000 1960 78  -.0013738747   21760.75   .010518823 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   29294.92    .009577673
    10006  74               .015625    .007948509000000001  69.1275     39000 1960 78    .010410005  34424.586    .01874654 3854004.0653224494 .3777748265532016 1.8997299301524146e-07 -1120.2025     .00515262
    10006  76  -.015384615398943424            -.001701185   68.064    -28800 1960 78  -.0040189917 -34750.324   -.01934587 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -13310.79   -.007264718
    10006  77              -.015625            .0008937427  67.0005    -23625 1960 78    .001774978 -26488.934  -.014807622 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -30465.77    -.01126336
    10006  82  .0026455025654286146             .007862628 67.17775     28425 1960 78    .004329341   8040.855  -.004967007 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  11739.702 -.00051755883
    10006  83  .0052770450711250305   .0043164810000000005 67.53225   33337.5 1960 78   .0020998297   30009.98   .004381898 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   25244.75  -.0012173032
    10006  87   .007874015718698502  -.0022199610000000003   68.064     14400 1960 78  -.0032162194  19607.717   .006923646 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   26795.32    .005364086
    10006  90  -.013020833022892475            -.006133015 67.17775    -18950 1960 78   -.005248301  -15210.27   -.01047212 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   1276.974   -.008148617
    10006  91 -.0052770450711250305             .002010226 66.82325  -14137.5 1960 78   .0021451344 -13640.546  -.003723557 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -22404.86  -.0034960876
    10006  94  -.007957560010254383             .001829586  66.2915    -60775 1960 78  -.0003689644  -71008.67  -.010087335 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -59353.01    .005671555
    10006  95   -.01604278013110161             .005499762   65.228    -27600 1960 78    .004977635  -23363.05  -.016518638 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -46783.82   -.013484857
    10006  96   .029891304671764374    .010100050000000001 67.17775   42637.5 1960 78    .007292831    43306.3    .02600555 3854004.0653224494 .3777748265532016 1.8997299301524146e-07    14530.9     .01890923
    10006  97   .018469657748937607            .0007903875  68.4185     33775 1960 78   .0010526361   31942.96   .020925364 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  29718.137    .012003698
    10006  98    .01295336801558733  -.0019942930000000003 69.30475      9775 1960 78  -.0004600182   22206.28    .01467619 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  11547.912    .008908558
    10006 101  .0025575447361916304            -.003782535   69.482     49000 1960 78   -.002838046   45119.13  .0016729067 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   59937.84   -.004941728
    10006 102   .005102040711790323   .0022702160000000002  69.8365     19700 1960 78    .002604394  2366.2268      .004413 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   9662.655    .003668647
    10006 103  -.005076142027974129           -.0001411817   69.482    -34300 1960 78   -.001303675  -41622.79  -.006691026 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -29275.63   .0033235585
    10006 104  -.005102040711790323              .00266341  69.1275    -39000 1960 78     .00286117 -36603.582  -.005006331 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -50026.96   .0007707733
    10006 108  -.015384615398943424             .002326248   68.064    -96000 1960 78   .0014948195  -93188.61  -.016610187 3854004.0653224494 .3777748265532016 1.8997299301524146e-07 -101761.04   .0017539985
    10006 110              -.015625             -.01157072  67.0005    -23625 1960 78   -.011795078 -21174.014  -.015021278 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   21833.28   -.007146626
    10006 111   -.01587301678955555   .0033597370000000002   65.937    -23250 1960 78    .005178119  -11609.21  -.012758518 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -43206.49   -.015623744
    10006 112  -.005376344081014395             -.00264063  65.5825    -32375 1960 78    -.00668199  -27307.29  -.006414204 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -6622.585     .00233559
    10006 116  -.029729729518294334             .003681965 63.63275  -31412.5 1960 78    .004456632 -20040.854   -.02818935 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -48588.38    -.02760611
    10006 117  -.002785515272989869            -.000160142  63.4555    -49225 1960 78   -.005347456  -46541.31   -.01073303 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -28615.88    .008076211
    10006 118   -.03910614550113678   -.008444440000000001   60.974    -43000 1960 78   -.006779317 -35120.582  -.033017814 3854004.0653224494 .3777748265532016 1.8997299301524146e-07 -16872.486    -.02987313
    10006 119   .011627906933426857  -.0034230890000000003   61.683     34800 1960 78   -.005557564    47712.8   .010494353 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   56218.88     .00466327
    10006 122    -.0517241396009922   -.004886028000000001  58.4925    -78375 1960 78   -.004265933  -64674.65   -.04449325 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -61934.08    -.03782614
    10006 123   .027272727340459824              .01375666 60.08775     76275 1960 78    .010439043   87564.59   .025926527 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  36042.887    .006694213
    10006 124   .044247787445783615             .004162762  62.7465    119475 1960 78  .00033154085  107927.47    .04046549 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  118197.24    .023619235
    10006 125  -.005649717524647713  -.0031448550000000002   62.392   -127600 1960 78   .0012142352  -103508.1   .005481715 3854004.0653224494 .3777748265532016 1.8997299301524146e-07 -132279.67     .01355532
    10006 126   .014204545877873898  -.0009699778000000001 63.27825     53550 1960 78   .0006465707    70870.6   .015749238 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   51058.11   .0004967874
    10006 129  -.014005602337419987   .0010128420000000001   62.392    -39600 1960 78     .00147494  -75306.04  -.016769862 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -45284.43 -.00025668385
    10006 131  -.019886363297700882             .002443152 61.15125  -38812.5 1960 78   .0020766105  -42512.45   -.02053308 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -46815.77   -.012594637
    10006 132  -.011594203300774097             .005493488 60.44225    -34100 1960 78   .0023048036  -46070.93   -.01682778 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -42982.72   -.003712668
    10006 133    .03812316805124283             .009154771  62.7465     17700 1960 78    .006925819  20973.846   .033753425 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -8992.133    .031522304
    10006 136   .005649717524647713  -.0005380566000000001   63.101     71200 1960 78   .0001283009   85029.63   .009298242 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   70705.53   -.010552084
    10006 137    .01123595517128706             .003900854    63.81      9000 1960 78    .004848164  19952.186    .01165658 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -9684.842    .005614065
    10006 139   .008333333767950535             .004453865 64.34175   31762.5 1960 78    .003502887   7652.248   .005869414 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   18262.36    .005556311
    10006 140    .01652892492711544             .002158872 65.40525     46125 1960 78   .0006872662    25052.8   .011355746 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   43476.27    .011509937
    10006 143    .01084010861814022  -.0006362553000000001 66.11425   32637.5 1960 78  .00004285305  35861.746    .00978411 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  32472.344   .0040111565
    10006 144  -.013404825702309608           -.0004946725 64.34175  -22687.5 1960 78   .0005945981   -26684.3   -.00966886 3854004.0653224494 .3777748265532016 1.8997299301524146e-07 -24979.084   -.008560153
    10006 145  -.011019283905625343           -.0001997464 63.63275    -26925 1960 78  -.0012905566  -34328.76  -.013552455 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -21951.19  -.0040102066
    10006 147  -.011142061091959476            .0008128089 62.92375    -26625 1960 78 -.00028792126 -38145.063     -.013053 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -25515.35    -.00378676
    10006 151   .014084506779909134             .001632732    63.81     45000 1960 78   .0004291791   50398.68   .014456122 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   43345.94   .0043479865
    10006 152  -.013888888992369175             .001665838 62.92375    -35500 1960 78   .0020317275  -24593.27   -.00992675 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -43330.29   -.009984368
    10006 153  -.014084506779909134             .003676377  62.0375    -48125 1960 78     .00238638  -42497.28  -.014513626 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -57322.12   -.006912685
    10006 154  -.008571428246796131   .0020684830000000003 61.50575    -34700 1960 78  -.0003350036  -48444.18   -.01361484 3854004.0653224494 .3777748265532016 1.8997299301524146e-07 -33408.895   .0007582142
    10006 157  -.014409221708774567              .01115093  60.6195    -76950 1960 78    .010114383  -78161.59  -.015924335 3854004.0653224494 .3777748265532016 1.8997299301524146e-07 -115930.88   -.003381589
    10006 158    .02923976629972458             .009586959   62.392     61600 1960 78    .005396535   61448.21    .02384328 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   40801.73     .01552759
    10006 159                .03125             .007508365 64.34175   40837.5 1960 78    .007224412      34588    .02885155 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  12994.586    .021950014
    10006 160  -.005509641952812672             .001137484 63.98725  -40612.5 1960 78     .00148009 -34233.242   -.00598236 3854004.0653224494 .3777748265532016 1.8997299301524146e-07  -46316.77  .00043460875
    10006 161   .008310249075293541            .0006981866   64.519     27300 1960 78  .00026311268   13569.25  .0028344374 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   26285.96    .005633061
    10006 164     .0714285746216774            .0007115134  69.1275    170625 1960 78  -.0004563272  140860.67    .06400865 3854004.0653224494 .3777748265532016 1.8997299301524146e-07   172383.7     .04484124
    10006 165  -.025641025975346565 .000030839989999999995   67.355    -90250 1960 78   .0042901947  -81046.35  -.016732754 3854004.0653224494 .3777748265532016 1.8997299301524146e-07 -106784.43   -.011865135
    end
    format %td date

    Comment


    • #3
      The reason you are getting the error message about variable mydate already existing is due to a subtle bug in your program. You named your program irf. Bad choice, because not only is there a Stata program -irf-, but you actually try to call it inside your program irf. Stata can't cope with two programs having the same name being used at the same time. When you define your program irf, Stata will no longer recognize the official -irf- program. So when you reach the command -irf create myirf,...-, within your program irf, Stata goes back and (recursively*) calls your program -irf- which starts by trying to again create variable mydate. But variable mydate really does exist there because you created it in the original call to your program -irf-. So this problem can be resolved by renaming your program to something else, something that is not the name of any other program that might be needed, and preferrably does not use the name of any official Stata program. Maybe call it -my_irf_prog-

      I can next respond to question 2 in #2.

      To save the current value of id in myirfs.dta, you are on the right track with your -gen id = `id'- command. The problem is that you never define local macro id, so the command will just throw a syntax error. The way to define local macro id is to do so as soon as you enter your program.
      Code:
      program my_irf_prog
      local id = id[1]
      gen mydate = _n
      // AND SO ON
      I have not delved deeply into other aspects of your program. But these remarks should resolve the particular problems I have responded to. I will note in passing that -duplicated- is not a valid Stata command; I imagine you meant -duplicates-. And I would also add that using -force- options is asking for trouble. I can't tell if, in this situation, you will get bitten by that, but a wiser course is to first -drop- any variables that can correctly vary across different observations with the same value of id, but are not relevant in the following computations, and then do an un-force-d -duplicates drop-. That way if something has gone awry that causes some thing that should be a constant not to be (or if you have misunderstood what should be a constant) Stata will stop you from blundering on with lost and incorrect information. Remember: error messages are your friends. They keep you from doing things that are inappropriate to the current state of your data. You should never try to suppress or ignore them: welcome them, embrace them, heed them, and fix, do not cover up, errors.

      I cannot respond to the other questions in your post because they require knowledge of the Stata -irf- program and -var-. Of the former I literally know nothing, and of the latter very little.

      *Added: You are actually quite lucky that you got this error about mydate already existing. If that command weren't there, you would be setting up an infinite sequence of recursive calls to your program -irf-, which would culminate in the error message "system limit exceeded - see manual," and I suspect you would have a hard time figuring out what went wrong. The fortuitous creation of variable mydate at the beginning of the call broke off that sequence and left you with an error message that was actually helpful in resolving the problem.
      Last edited by Clyde Schechter; 30 Oct 2022, 11:37.

      Comment


      • #4
        Originally posted by Clyde Schechter View Post
        The reason you are getting the error message about variable mydate already existing is due to a subtle bug in your program. You named your program irf. Bad choice, because not only is there a Stata program -irf-, but you actually try to call it inside your program irf. Stata can't cope with two programs having the same name being used at the same time. When you define your program irf, Stata will no longer recognize the official -irf- program. So when you reach the command -irf create myirf,...-, within your program irf, Stata goes back and (recursively*) calls your program -irf- which starts by trying to again create variable mydate. But variable mydate really does exist there because you created it in the original call to your program -irf-. So this problem can be resolved by renaming your program to something else, something that is not the name of any other program that might be needed, and preferrably does not use the name of any official Stata program. Maybe call it -my_irf_prog-

        *Added: You are actually quite lucky that you got this error about mydate already existing. If that command weren't there, you would be setting up an infinite sequence of recursive calls to your program -irf-, which would culminate in the error message "system limit exceeded - see manual," and I suspect you would have a hard time figuring out what went wrong. The fortuitous creation of variable mydate at the beginning of the call broke off that sequence and left you with an error message that was actually helpful in resolving the problem.
        Dear Clyde, thanks for your information. Now I have fixed the program
        Code:
        program my_irf
            local gid = id[1]
            gen mydate =_n
            xtset id mydate
            matrix A1 = (1,0,0 \ .,1,0 \ .,.,1)
            matrix B1 = (.,0,0 \ 0,.,0 \ 0,0,.)
            svar vwretd sdvol RET, lags(1/5) aeq(A1) beq(B1)
            irf create myirf, set(myirfs, replace) step(15)
            save complete.dta,replace
            use "myirfs.irf",clear
            gen id = `gid'
            egen response_rm_1_1 = total(cirf) if step == 15 & response =="vwretd"
            egen response_rm_2_1 = total(cirf) if step == 15 & response =="vwretd" & impulse =="vwretd"
            egen response_rm_3_1 = total(cirf) if step == 15 & response =="RET" & impulse =="vwretd"
            egen response_rm_1 = mean(response_rm_1_1)
            egen response_rm_2 = mean(response_rm_2_1)
            egen response_rm_3 = mean(response_rm_3_1)
            drop response_rm_1_1 response_rm_2_1 response_rm_3_1
            egen response_x_1_1 = total(cirf) if step == 15 & response =="sdvol"
            egen response_x_2_1 = total(cirf) if step == 15 & response =="sdvol" & impulse =="sdvol"
            egen response_x_3_1 = total(cirf) if step == 15 & response =="RET" & impulse =="sdvol"
            egen response_x_1 = mean(response_x_1_1)
            egen response_x_2 = mean(response_x_2_1)
            egen response_x_3 = mean(response_x_3_1)
            drop response_x_1_1 response_x_2_1 response_x_3_1
            egen response_r_1_1 = total(cirf) if step == 15 & response =="RET"
            egen response_r_2_1 = total(cirf) if step == 15 & response =="RET" & impulse =="RET"
            egen response_r_3_1 = total(cirf) if step == 15 & response =="RET" & impulse =="RET"
            egen response_r_1 = mean(response_r_1_1)
            egen response_r_2 = mean(response_r_2_1)
            egen response_r_3 = mean(response_r_3_1)
            drop response_r_1_1 response_r_2_1 response_r_3_1
            keep id response_rm_1 response_rm_2 response_rm_3 response_x_1 response_x_2 response_x_3 response_r_1 response_r_2 response_r_3
            duplicates drop id, force
            save "myirfs.irf", replace
            use complete.dta, clear
            merge m:1 id using myirfs.irf
            drop mydate _merge
        end
        
        forval y = 1960/2015{
            use `y'.dta,clear
            keep PERMNO date RET vwretd size sdvol year id
            runby my_irf, by(id) verbose
            drop date RET vwretd sdvol
            duplicates drop id, force
            save `y'_irf.dta, replace
        }
        -runby- now can execute the program named my_irf successfully by each id in each year-file. The yearly file stores the daily data for each stock available in CRSP database (sample can be found in data in #1), average around 4000 stocks per year.
        Even though now -runby- works, and generated yearly irf file, with names year_irf.dta. But it takes so much time to process. I spent already 12 hr and only processed 10 yearly files. Is it possible that I can accelerate it?
        Last edited by Wen-Hung Hsu; 31 Oct 2022, 05:20.

        Comment


        • #5
          Because I do not understand what this code does or how it works, I can't be very helpful here. A couple of things strike me as likely bottlenecks, but I don't know if there is any way to eliminate them while still having the code do what you need it to do. Here are my impressions, for what, if anything, they are worth.

          1. The bulk of program my_irf consists of reading in the file myirfs.irf, doing some calculations, and then saving that file. Do you really need to save that file each time? It looks to me like it just gets over-written early in the next iteration of the program anyway. Writing a file to disk is very slow, and the less often you do it the better. All of the results myirfs.irf contains, I think, will also appear in the final results data set when -runby- finishes, because another thing that program my_irf does is merge those results in with the other data, which is another reason that -save-ing it looks pointless.

          2. Many of the computations you do in program my_irf have an -if- clause. -if- clauses are very slow and the whole point of using -runby- is to minimize or avoid their use entirely. Now, it doesn't look to me like you can actually eliminate those -if- clauses. But you may be able to reduce the size of the data set they apply to. It seems that for the calculations you are interested only in the observations where step == 15. So why not first -keep if step == 15-. This will "debulk" the data so that the -if-clauses will run faster because there are fewer observations that need to be evaluated.

          3. It appears to me that all of the commands -egen response_whatever = mean(response_whatever_1)- are superfluous. That's because in all those instances, response_whatever_1 was itself the result of an -egen- command with the result that the value it calculates is exactly the same in every observation, except those where it is missing. Consequently, the mean is just that value. This implies that instead of two steps, you can do it all in a single step: -egen response_whatever = total(cond(response == "something" & impulse == "something_else", cirf, .)- This also eliminates the need to -drop response_whatever_1-, because you never create those variables in the first place. If nothing else, this will reduce the number of lines of code by about 50% as well as speeding up the execution of those lines by using a smaller data set for the calculations. Here's a demonstration of the technique I'm proposing using the auto.dta, and demonstrating that it produces the same results:
          Code:
          . clear*
          
          . sysuse auto
          (1978 automobile data)
          
          . gen long obs_no = _n
          
          .
          . //  CREATE A DATA SET RESTRICTED TO FOREIGN = 1
          . frame put _all if foreign == 1, into(foreign_cars)
          
          .
          . //  THIS IS LIKE THE ORIGINAL CALCULATIONS
          . egen wanted1_1 = total(weight) if foreign == 1 & rep78 == 3
          (71 missing values generated)
          
          . egen wanted1 = mean(wanted1_1)
          
          .
          . //  THIS IS THE MODIFIED CALCULATION USING A SINGLE LINE OF CODE
          . //  ON THE REDUCED DATA SET
          . frame foreign_cars {
          .     egen wanted1 = total(cond(rep78 == 3, weight, .))
          . }
          
          .
          . //  VERIFY THAT THE RESULTS ARE THE SAME
          . frlink 1:1 obs_no, frame(foreign_cars)
            (52 observations in frame default unmatched)
          
          . assert wanted1 == frval(foreign_cars, wanted1) if !missing(foreign_cars)
          4. Once you are satisfied that program -my_irf- is working as it should, you should eliminate the -verbose- option from the -runby- command. Writing out all of that output is also slowing things down and serves no purpose. The -verbose- option is best used only while you are still in debugging mode. Once you are satisfied your code is correct, you should get rid of it. I would then add, instead, the -status- option so that -runby- will give you a periodic progress report and estimate of time remaining.

          Finally, I need to say that there are some things that just take a long time and you need to be patient. In my own work, I am quite accustomed to programs that take up to 2 weeks to run. I really only start to balk at things that will run to months because it is hard to keep a computer running that long without interruption.

          Comment


          • #6
            Originally posted by Clyde Schechter View Post
            1. The bulk of program my_irf consists of reading in the file myirfs.irf, doing some calculations, and then saving that file. Do you really need to save that file each time? It looks to me like it just gets over-written early in the next iteration of the program anyway. Writing a file to disk is very slow, and the less often you do it the better. All of the results myirfs.irf contains, I think, will also appear in the final results data set when -runby- finishes, because another thing that program my_irf does is merge those results in with the other data, which is another reason that -save-ing it looks pointless.
            Thank you Clyde. For the 2-4 solutions I have amend my code.
            However, for solution 1, I have some concerns. The reason that I save the irf file and complete file is because I want to set the complete the master file when merging. But through the work flow of the coding, it needs to open the irf file to do some calculation and then open the original data to merge the results from irf file. That is the reason I have some many -save- in the program. Or is there is a workaround that I do not need to save the files in preparation for -merge-?

            Comment


            • #7
              But couldn't you just do:
              Code:
              program my_irf
                  local gid = id[1]
                  gen mydate =_n
                  xtset id mydate
                  matrix A1 = (1,0,0 \ .,1,0 \ .,.,1)
                  matrix B1 = (.,0,0 \ 0,.,0 \ 0,0,.)
                  svar vwretd sdvol RET, lags(1/5) aeq(A1) beq(B1)
                  irf create myirf, set(myirfs, replace) step(15)
                  save complete.dta,replace
                  use "myirfs.irf",clear
                  gen id = `gid'
                  // EGEN COMMANDS GO HERE
                  duplicates drop id, force
                  merge 1:m id using complete
                  drop mydate _merge
              end
              which enables you to merge the augmented myirfs.irf data into complete.dta without saving myirfs.irf. After all, after you save it, -runby- is going to reun program my_irf again, and then the -irf create- command is just going to clobber the myirfs.irf file you just saved, right?

              Comment


              • #8
                Originally posted by Clyde Schechter View Post
                But couldn't you just do:
                which enables you to merge the augmented myirfs.irf data into complete.dta without saving myirfs.irf. After all, after you save it, -runby- is going to reun program my_irf again, and then the -irf create- command is just going to clobber the myirfs.irf file you just saved, right?
                I finally understand your point. You are right: with -runby-, it would save the results back to the original data according to id. Therefore indeed there is no reason to do the -save- and reverse the position of Master file or merging file. I was stuck with normal procedure, forgetting how -runby- works.

                Comment


                • #9
                  And for those who are also struggled with the reduced VAR/ structured VAR. I found a useful reference:
                  https://blog.stata.com/2016/08/09/ve...ions-in-stata/

                  What I was trying to do is to set the A and B matrix and apply them in -svar.
                  However, it seems that -var- is already embedded with the A down triangle matrix. Therefore the irf results from -var is identical with results from -svar-. The only difference is that Stata process -var- quicker.

                  Comment


                  • #10
                    Hi!

                    svar leaves you the opportunity to specify A and B matrices the way you wish, it does not have to be a cholesky decomposition by itself. For that you need to put restrictions as you wish.

                    Comment


                    • #11
                      Originally posted by Morad Bali View Post
                      Hi!

                      svar leaves you the opportunity to specify A and B matrices the way you wish, it does not have to be a cholesky decomposition by itself. For that you need to put restrictions as you wish.
                      Thank you for the clarification. Yes, I forgot -svar- is more customized according to the A and B matrices. I was focusing on my project, where normal cholesky decompossition is used ( the upper variables affect the lower variables, but not vise versa.). But to be honest, the process speed of -svar- compared with -var- when both using Cholesky decomposition is so slow.

                      Comment


                      • #12
                        Originally posted by Wen-Hung Hsu View Post

                        Thank you for the clarification. Yes, I forgot -svar- is more customized according to the A and B matrices. I was focusing on my project, where normal cholesky decompossition is used ( the upper variables affect the lower variables, but not vise versa.). But to be honest, the process speed of -svar- compared with -var- when both using Cholesky decomposition is so slow.
                        So just to be clear, with the same data you obtained the same results when using svar and var command? Only difference lays in the processing time?

                        Comment


                        • #13
                          Originally posted by Morad Bali View Post

                          So just to be clear, with the same data you obtained the same results when using svar and var command? Only difference lays in the processing time?
                          Yes. Given the same data,
                          -var-
                          Code:
                          var vwretd sdvol RET, lags(1/5) 
                          irf create myirf, set(myirfs, replace) step(15)
                          and -svar-
                          Code:
                          matrix A1 = (1,0,0 \ .,1,0 \ .,.,1)
                          matrix B1 = (.,0,0 \ 0,.,0 \ 0,0,.)
                          svar vwretd sdvol RET, lags(1/5) aeq(A1) beq(B1)
                          irf create myirf, set(myirfs, replace) step(15)
                          both generate exactly the same results. The setting of AB matrices in -svar- is so-called Cholesky decomposition I think.

                          The only difference is the processing time. For var, I processed 55 years for around 1 whole day, while for svar, I processed 7 years for half day. The processing efficiency of svar is miserable.

                          Comment


                          • #14
                            Originally posted by Wen-Hung Hsu View Post

                            Yes. Given the same data,
                            -var-
                            Code:
                            var vwretd sdvol RET, lags(1/5)
                            irf create myirf, set(myirfs, replace) step(15)
                            and -svar-
                            Code:
                            matrix A1 = (1,0,0 \ .,1,0 \ .,.,1)
                            matrix B1 = (.,0,0 \ 0,.,0 \ 0,0,.)
                            svar vwretd sdvol RET, lags(1/5) aeq(A1) beq(B1)
                            irf create myirf, set(myirfs, replace) step(15)
                            both generate exactly the same results. The setting of AB matrices in -svar- is so-called Cholesky decomposition I think.

                            The only difference is the processing time. For var, I processed 55 years for around 1 whole day, while for svar, I processed 7 years for half day. The processing efficiency of svar is miserable.
                            Out of curiosity, did you try to run and compare IRF and FEVD?

                            Comment


                            • #15
                              Originally posted by Morad Bali View Post

                              Out of curiosity, did you try to run and compare IRF and FEVD?
                              IRF is the same. What is FEVD?

                              Comment

                              Working...
                              X