Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Synth command is producing missing values for its results

    Dear All,

    I am trying to estimate synthetic control difference-in-difference case studies. The unit of observation is state-quarter, with 34 states and 26 quarters. 15 of the states are treated, with varied treatment periods spread between the 5th-24rth quarters. The remaining 19 states belong to the donor group for creating the synthetic controls. So eventually, I will have 15 case studies, one for each state.

    I have tried the following code which has given me missing results:
    Code:
    local outcome narc /*otherreason misuse suicide reasonunknown nomineff modeff majeff de
    > ath effmiss */
    
    . local controlvars  age015 age1624 age2534 age3544 age4554 age5564 age65plus agemiss male pop
    
    .
    . collapse  (mean) `outcome' `controlvars' qtrtreat , by (trunit qtr)
    
    .
    . tsset trunit qtr
    panel variable:  trunit (strongly balanced)
    time variable:  qtr, 1 to 26
    delta:  1 unit
    
    .
    . foreach y in `outcome' {
    2. forvalues lname = 1/15 {
    3. preserve
    4. keep if (trunit==`lname'trunit>15)
    5. egen maxp_`lname'=max(qtrtreat)
    6. local p=maxp_`lname'
    7. display `p'
    8. local p1 = `p' - 1
    9. display `p1'
    10. /*qui*/ synth `y' `controlvars' `y'(1(1)`p1') , trunit(`lname') trperiod(`p') fig ke
    > ep(synthetic_casestudy_`y'_`lname', replace)       
    11. restore
    12. }
    13. }
    (364 observations deleted)
    5
    4
    
    Synthetic Control Method for Comparative Case Studies
    
    
    First Step: Data Setup
    
    
    Data Setup successful
    
    Treated Unit: 1
    Control Units:  16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
    
    Dependent Variable: narc
    MSPE minimized for periods: 1 2 3 4
    Results obtained for periods: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
    24 25 26
    
    Predictors: age015 age1624 age2534 age3544 age4554 age5564 age65plus
    agemiss male pop narc(1(1)4)
    
    Unless period is specified
    predictors are averaged over: 1 2 3 4
    
    
    Second Step: Run Optimization
    
    
    Optimization done
    
    
    Third Step: Obtain Results
    
    Loss: Root Mean Squared Prediction Error
    
    
    RMSPE          .
    
    
    Unit Weights:
    
    
    Co_No  Unit_Weight
    
    16            
    17            
    18            
    19            
    20            
    21            
    22            
    23            
    24            
    25            
    26            
    27            
    28            
    29            
    30            
    31            
    32            
    33            
    34            
    
    
    Predictor Balance:
    
    
    Treated  Synthetic
    
    age015   .1887048          .
    age1624   .2452776          .
    age2534    .183628          .
    age3544   .1065218          .
    age4554   .0979623          .
    age5564   .0652333          .
    age65plus   .0674047          .
    agemiss   .0452675          .
    male   .4161599          .
    pop   11544.82          .
    narc(1(1)4)          1          .
    
    
    (364 observations deleted)
    I am not really sure why synth is unable to estimate the RMSPE. The problem may simply be with my loops....

    I will be very grateful for any help I can get.
    Sincerely,
    Sumedha.

  • #2
    Try running it without the loop first, specifying things by hand. Once that works, go back to the loop, but use display before the synth step to make sure every local looks correct.

    Another option is to use synth_runner, which is a wonderful wrapper for synth that handles multiple treated units for you.

    Comment


    • #3
      Thank you for your suggestions, Prof. Masterov. I have tried to remove all the loops and go as manual as possible, but with no luck:
      local outcome narc /*otherreason misuse suicide reasonunknown nomineff modeff majeff de
      > ath effmiss */

      . local controlvars age015 age1624 age2534 age3544 age4554 age5564 age65plus agemiss mal
      > e pop

      .
      . collapse (mean) `outcome' `controlvars' qtrtreat , by (trunit qtr)

      .
      . keep if trunit==1|trunit>15
      (364 observations deleted)

      .
      . egen maxp_1=max(qtrtreat)

      . local p=maxp_1

      . display `p'
      5

      . local p1=`p'-1

      . display `p1'
      4

      . tsset trunit qtr
      panel variable: trunit (strongly balanced)
      time variable: qtr, 1 to 26
      delta: 1 unit

      .
      . synth narc `controlvars' narc(1) narc(2) narc(3) narc(4), trunit(1) trperiod(5) fig kee
      > p(synthetic_casestudy_narc_1, replace)
      -----------------------------------------------------------------------------------------
      Synthetic Control Method for Comparative Case Studies
      -----------------------------------------------------------------------------------------

      First Step: Data Setup
      -----------------------------------------------------------------------------------------
      -----------------------------------------------------------------------------------------
      Data Setup successful
      -----------------------------------------------------------------------------------------
      Treated Unit: 1
      Control Units: 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
      -----------------------------------------------------------------------------------------
      Dependent Variable: narc
      MSPE minimized for periods: 1 2 3 4
      Results obtained for periods: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
      24 25 26
      -----------------------------------------------------------------------------------------
      Predictors: age015 age1624 age2534 age3544 age4554 age5564 age65plus
      agemiss male pop narc(1) narc(2) narc(3) narc(4)
      -----------------------------------------------------------------------------------------
      Unless period is specified
      predictors are averaged over: 1 2 3 4
      -----------------------------------------------------------------------------------------

      Second Step: Run Optimization
      -----------------------------------------------------------------------------------------
      -----------------------------------------------------------------------------------------
      Optimization done
      -----------------------------------------------------------------------------------------

      Third Step: Obtain Results
      -----------------------------------------------------------------------------------------
      Loss: Root Mean Squared Prediction Error

      ---------------------
      RMSPE | .
      ---------------------
      -----------------------------------------------------------------------------------------
      Unit Weights:

      -----------------------
      Co_No | Unit_Weight
      ----------+------------
      16 |
      17 |
      18 |
      19 |
      20 |
      21 |
      22 |
      23 |
      24 |
      25 |
      26 |
      27 |
      28 |
      29 |
      30 |
      31 |
      32 |
      33 |
      34 |
      -----------------------
      -----------------------------------------------------------------------------------------
      Predictor Balance:

      ------------------------------------------------------
      | Treated Synthetic
      -------------------------------+----------------------
      age015 | .1887048 .
      age1624 | .2452776 .
      age2534 | .183628 .
      age3544 | .1065218 .
      age4554 | .0979623 .
      age5564 | .0652333 .
      age65plus | .0674047 .
      agemiss | .0452675 .
      male | .4161599 .
      pop | 11544.82 .
      narc(1) | 1 .
      narc(2) | 1 .
      narc(3) | 1 .
      narc(4) | 1 .
      ------------------------------------------------------
      -----------------------------------------------------------------------------------------

      .
      end of do-file
      Here is a dataex of the data :

      Code:
      * Example generated by -dataex-. To install: ssc install dataex
      clear
      input float(qtr trunit narc age015 age1624 age2534 age3544 age4554 age5564 age65plus agemiss male) double pop float(qtrtreat maxp_1)
       1  1 1  .1904762 .21693122  .1851852   .1005291 .12169312  .07936508  .04232804  .06349207  .3703704 11544.824000000017 5 5
       2  1 1       .24 .20666666 .19333333  .09333333        .1 .033333335        .08  .05333333       .44 11544.824000000017 5 5
       3  1 1 .15172414 .28965518 .16551724  .13103448 .06896552  .08275862  .07586207  .03448276  .4137931 11544.824000000017 5 5
       4  1 1 .17261904 .26785713  .1904762  .10119048 .10119048 .065476194 .071428575 .029761905  .4404762 11544.824000000019 5 5
       5  1 1 .15384616  .3076923  .2167832  .07692308  .0909091 .062937066 .062937066  .02797203  .4755245 11550.838999999978 5 5
       6  1 1 .15441176  .2132353  .1617647  .13970588 .10294118  .11029412  .08088236 .036764707  .4191177 11550.838999999982 5 5
       7  1 1  .1769231  .2846154 .15384616   .1076923 .07692308  .07692308  .06153846  .06153846  .3615385 11550.838999999984 5 5
       8  1 1 .22292994  .2611465 .12738854  .10191083 .15286624  .06369427  .04458599  .02547771  .4585987 11550.838999999973 5 5
       9  1 1 .23776224  .2027972 .12587413   .0979021 .08391608  .13286713  .04195804  .07692308  .4825175  11570.02200000004 5 5
      10  1 1 .16197184 .26760563 .13380282  .14788732 .08450704   .0915493  .07042254  .04225352 .41549295 11570.022000000039 5 5
      11  1 1 .20437956 .23357664 .15328467  .13138686 .08029197  .08029197   .0729927  .04379562 .50364965 11570.022000000037 5 5
      12  1 1  .1983471  .2479339  .1818182  .14876033 .07438017   .0661157  .03305785  .04958678  .4214876 11570.022000000026 5 5
      13  1 1 .15686275  .2254902 .14705883  .19607843 .04901961  .05882353  .10784314  .05882353  .3921569 11594.408000000029 5 5
      14  1 1  .1567164 .23880596 .17910448  .09701493  .1641791  .08208955  .04477612  .03731343  .4253731 11594.408000000034 5 5
      15  1 1  .1551724  .1637931 .18103448   .1637931 .11206897   .0775862  .12068965  .02586207  .4396552 11594.408000000032 5 5
      16  1 1 .14864865  .2837838 .08108108   .1081081  .1081081   .0945946   .0945946  .08108108 .41216215 11594.408000000036 5 5
      17  1 1  .2184874  .1764706 .14285715  .05042017 .11764706  .16806723 .067226894  .05882353  .4957983 11605.090000000011 5 5
      18  1 1  .1743119  .1743119 .13761468   .0825688 .11009175  .11926606  .12844037  .07339449 .42201835 11605.090000000006 5 5
      19  1 1 .12903225 .20430107   .172043   .0967742 .13978495  .04301075  .11827957   .0967742  .4408602 11605.089999999991 5 5
      20  1 1 .14049587  .2231405  .2231405   .1322314 .05785124  .09917355  .09917355  .02479339   .446281 11605.090000000013 5 5
      21  1 1 .11206897 .20689656  .1637931  .11206897 .11206897  .10344828  .12068965  .06896552  .4913793  11614.37299999999 5 5
      22  1 1 .11607143  .2142857 .14285715  .14285715 .11607143 .071428575   .1517857  .04464286  .4196429 11614.372999999994 5 5
      23  1 1  .1825397  .1904762 .16666667  .15873016 .07936508   .1031746   .1031746 .015873017  .4365079 11614.372999999985 5 5
      24  1 1 .20212767 .11702128 .19148937  .14893617 .13829787  .08510638  .05319149  .06382979  .4574468 11614.373000000012 5 5
      25  1 1      .125 .23958333    .15625  .11458334 .10416666       .125     .09375  .04166667        .5  11658.60900000002 5 5
      26  1 1 .11864407  .2457627 .18644068  .11864407 .08474576  .09322034  .11864407 .033898305  .4745763 11658.609000000006 5 5
       1 16 1 .14492753 .28985506  .2463768 .072463766 .11594203  .10144927 .014492754 .014492754  .3623188 3065.2229999999986 0 5
       2 16 1  .1147541 .32786885 .14754099  .09836066  .1147541 .032786883   .1147541  .04918033  .3606557 3065.2229999999986 0 5
       3 16 1 .09433962 .26415095 .13207547   .1509434  .2264151   .0754717 .018867925  .03773585 .41509435 3065.2229999999986 0 5
       4 16 1  .1794872  .3333333 .12820514  .07692308  .1025641  .05128205  .05128205  .07692308  .4615385 3065.2229999999986 0 5
       5 16 1 .11627907 .25581396 .11627907   .1860465 .09302326  .09302326  .09302326  .04651163  .3255814  3076.309999999998 0 5
       6 16 1 .13725491 .15686275 .19607843   .1764706 .07843138   .1764706  .03921569  .03921569  .4901961  3076.309999999998 0 5
       7 16 1 .15686275 .21568628 .15686275  .19607843 .07843138  .07843138  .09803922 .019607844  .4705882  3076.309999999998 0 5
       8 16 1 .04444445  .1777778 .24444444  .15555556 .11111111  .13333334  .13333334          0  .3555556  3076.309999999998 0 5
       9 16 1 .12765957 .17021276 .25531915  .10638298 .17021276  .10638298 .021276595  .04255319  .3829787  3091.929999999996 0 5
      10 16 1 .27450982 .15686275 .15686275  .19607843 .05882353  .03921569  .09803922 .019607844 .43137255 3091.9299999999957 0 5
      11 16 1        .2  .3111111 .11111111  .13333334 .06666667  .13333334  .02222222  .02222222  .4888889  3091.929999999996 0 5
      12 16 1 .17307693 .23076923 .21153846  .11538462 .07692308  .09615385  .05769231  .03846154  .4423077 3091.9299999999957 0 5
      13 16 1       .16       .22       .16         .2        .1        .06        .08        .02       .34 3108.0299999999993 0 5
      14 16 1  .1851852  .0925926 .24074075  .16666667 .14814815   .0925926  .05555556 .018518519  .3333333 3108.0299999999993 0 5
      15 16 1 .08510638 .25531915 .19148937  .14893617 .10638298  .12765957 .021276595  .06382979  .5744681 3108.0299999999993 0 5
      16 16 1 .09433962 .18867925 .16981132  .09433962  .1509434  .09433962  .16981132  .03773585  .4716981 3108.0299999999993 0 5
      17 16 1       .04       .22       .22        .22        .1        .08        .06        .06        .3 3121.9970000000017 0 5
      18 16 1 .13636364       .25 .13636364   .0909091 .13636364   .0909091  .13636364 .022727273  .3863636 3121.9970000000017 0 5
      19 16 1     .0625     .1875 .29166666  .16666667 .14583333  .04166667  .04166667      .0625     .4375 3121.9970000000017 0 5
      20 16 1        .1       .26       .14         .1       .16        .08        .14        .02       .38 3121.9970000000017 0 5
      21 16 1 .13157895  .2631579  .2368421  .10526316 .02631579  .15789473  .07894737          0 .28947368 3134.6929999999998 0 5
      22 16 1 .16666667  .2037037  .2037037  .12962963 .16666667  .05555556  .05555556 .018518519 .44444445 3134.6929999999998 0 5
      23 16 1 .19565217  .2173913 .19565217  .13043478 .06521739  .08695652  .08695652  .02173913 .45652175 3134.6929999999998 0 5
      24 16 1        .1        .3      .125       .125      .125       .175          0        .05       .55 3134.6929999999998 0 5
      25 16 1        .2       .28       .12        .08       .08        .12        .12          0        .4 3145.7110000000002 0 5
      26 16 1  .0882353  .2352941 .14705883   .0882353  .1764706  .05882353   .1764706 .029411765  .3529412 3145.7109999999993 0 5
       1 17 1 .16666667 .16666667 .16666667   .1764706 .13725491  .09803922 .068627454 .019607844  .3333333 6822.5200000000095 0 5
       2 17 1       .17       .18       .17        .15       .08        .12        .09        .04       .43 6822.5200000000095 0 5
       3 17 1 .12820514 .16666667  .2179487  .11538462 .12820514  .07692308   .0897436  .07692308  .3589744  6822.520000000007 0 5
       4 17 1  .1011236 .19101124  .1797753  .08988764 .16853933   .1011236   .1235955  .04494382  .4606742  6822.520000000008 0 5
       5 17 1 .12162162 .16216215 .24324325  .16216215 .08108108  .12162162  .04054054 .067567565  .3648649   6895.22600000001 0 5
       6 17 1 .11627907 .12790698 .19767442  .13953489 .10465116   .1627907  .13953489 .011627907 .29069766  6895.226000000012 0 5
       7 17 1 .23376623 .25974026  .1948052 .064935066 .07792208  .11688311  .03896104 .012987013  .4675325  6895.226000000011 0 5
       8 17 1 .10465116 .23255815  .1511628  .12790698 .11627907  .09302326  .12790698  .04651163  .3372093  6895.226000000012 0 5
       9 17 1       .15       .23       .13        .12       .18         .1        .05        .04       .45  6968.006000000009 0 5
      10 17 1 .13414635  .2682927 .18292683  .04878049  .1097561  .13414635  .08536585 .036585364  .4268293  6968.005999999999 0 5
      11 17 1 .11764706 .10588235  .2235294  .07058824 .10588235   .2352941  .05882353  .08235294  .3647059  6968.006000000001 0 5
      12 17 1  .1612903  .2096774  .1935484   .1935484 .11290322 .016129032  .08064516 .032258064  .3387097  6968.005999999995 0 5
      13 17 1 .13483146  .2247191 .13483146   .1011236  .1573034  .06741573   .1235955  .05617978  .4382023  7054.195999999997 0 5
      14 17 1  .1369863 .24657534 .10958904  .10958904  .0958904  .10958904   .0958904   .0958904  .4383562  7054.195999999997 0 5
      15 17 1 .16666667 .17857143 .11904762  .14285715  .0952381  .14285715  .08333334 .071428575  .3452381  7054.195999999998 0 5
      16 17 1  .1547619 .17857143 .11904762  .16666667 .10714286  .17857143  .08333334 .011904762  .4404762  7054.195999999998 0 5
      17 17 1 .12222222  .1777778  .1888889  .16666667 .13333334   .0888889         .1  .02222222  .3666667            7160.29 0 5
      18 17 1 .16666667  .1547619 .14285715  .05952381  .1904762  .14285715  .10714286 .035714287  .3452381  7160.289999999997 0 5
      19 17 1  .1857143        .2 .12857144  .11428571 .08571429  .15714286  .11428571 .014285714  .3857143  7160.289999999992 0 5
      20 17 1 .20454545 .17045455 .14772727  .14772727  .0909091   .0909091  .07954545  .06818182  .4318182  7160.289999999999 0 5
      21 17 1 .17567568  .1891892  .2162162  .12162162 .02702703  .16216215  .08108108  .02702703   .527027               7288 0 5
      22 17 1   .203125     .1875    .09375      .0625    .09375    .078125    .140625    .140625   .390625               7288 0 5
      23 17 1  .1612903  .0967742 .11290322   .1935484  .1612903  .08064516  .12903225  .06451613  .4032258               7288 0 5
      24 17 1 .15384616 .23076923        .2  .04615385 .12307692  .06153846  .15384616  .03076923  .4153846               7288 0 5
      25 17 1  .1038961 .11688311 .15584415   .0909091 .14285715  .16883117  .15584415 .064935066  .4675325  7405.743000000006 0 5
      26 17 1 .20338982 .16949153 .08474576  .13559322 .10169491  .16949153  .10169491 .033898305  .2881356  7405.743000000003 0 5
       1 18 1 .08333334       .25       .25  .16666667 .08333334  .08333334          0  .08333334        .5  685.4759999999998 0 5
       2 18 1  .0909091  .0909091 .27272728   .0909091  .1818182   .0909091          0   .1818182  .4545455  685.4759999999998 0 5
       3 18 1 .11111111 .44444445 .22222222          0 .11111111  .11111111          0          0  .5555556  685.4759999999999 0 5
       4 18 1 .27272728  .0909091 .27272728   .0909091 .27272728          0          0          0  .3636364  685.4759999999998 0 5
       5 18 1       .15       .15        .4        .15         0          0          0        .15        .4  702.0869999999998 0 5
       6 18 1        .6        .2         0         .2         0          0          0          0        .2            702.087 0 5
       7 18 1      .125       .25         0       .375      .125          0       .125          0      .375            702.087 0 5
       8 18 1 .11764706 .29411766 .11764706          0  .1764706  .11764706  .11764706  .05882353 .29411766  702.0869999999998 0 5
       9 18 1  .0909091         0  .1818182  .27272728  .0909091   .1818182          0   .1818182 .54545456  724.0190000000001 0 5
      10 18 1         0        .1        .4         .1        .2          0         .1         .1        .2  724.0190000000001 0 5
      11 18 1 .14285715 .14285715 .14285715          0  .4285714  .14285715          0          0  .5714286  724.0190000000001 0 5
      12 18 1      .125      .375      .125       .125      .125       .125          0          0       .25  724.0190000000001 0 5
      13 18 1 .08333334 .16666667 .16666667  .16666667 .08333334  .16666667          0  .16666667  .8333333  739.9040000000001 0 5
      14 18 1        .1        .2         0          0        .4         .1         .2          0        .2  739.9040000000001 0 5
      15 18 1 .15384616 .15384616 .15384616   .3076923 .23076923          0          0          0  .4615385  739.9040000000002 0 5
      16 18 1 .05263158 .15789473  .3157895  .10526316  .2105263  .05263158  .05263158  .05263158  .4210526  739.9040000000002 0 5
      17 18 1 .14285715         0  .8571429          0         0          0          0          0  .5714286            756.835 0 5
      18 18 1         0 .16666667         0   .3333333 .16666667          0  .16666667  .16666667  .8333333            756.835 0 5
      19 18 1         0  .4615385         0          0  .3846154  .07692308  .07692308          0  .3846154  756.8349999999999 0 5
      20 18 1 .14285715  .4285714  .2857143  .14285715         0          0          0          0  .2857143            756.835 0 5
      21 18 1        .2        .1        .1          0        .1         .3         .2          0        .3  757.9520000000001 0 5
      22 18 1         0  .6666667         0  .16666667         0          0  .16666667          0         0  757.9520000000001 0 5
      end
      It all seems sensible but of course its not. I will appreciate any help. Once I understand what is going on I will definitely learn synth_runner as well.

      Thanks,
      Sumedha.

      Comment


      • #4
        There is something funny with your narc variable. You can see that in the predictor balance output it is always one in treatment and missing in control (and it is always one in in your example data for all units and times). If there is no variation in the data, this method won't work when you try to pick weights where the lags are one of the things you hope to balance.

        Dropping the lagged narc variables from the set of predictors (and balancing the data), your code works on the toy data (see code below). It is good advice to start with the simplest model first and build your way up. In this case, that is the the model with just the control variables and no lags.

        I would take a look at the plot generated by:

        Code:
        xtiline narc, overlay
        It should not look like this:
        Click image for larger version

Name:	Screen Shot 2018-08-29 at 10.22.58 AM.png
Views:	1
Size:	36.1 KB
ID:	1460147


        In general, you should always look at plots or summary statistics of your data before running analyses.

        Here's Stata code showing my claim. Note that using the code tags (# rather than "), makes the output look nicer:

        Code:
        clear
        input float(qtr trunit narc age015 age1624 age2534 age3544 age4554 age5564 age65plus agemiss male) double pop float(qtrtreat maxp_1)
         1  1 1  .1904762 .21693122  .1851852   .1005291 .12169312  .07936508  .04232804  .06349207  .3703704 11544.824000000017 5 5
         2  1 1       .24 .20666666 .19333333  .09333333        .1 .033333335        .08  .05333333       .44 11544.824000000017 5 5
         3  1 1 .15172414 .28965518 .16551724  .13103448 .06896552  .08275862  .07586207  .03448276  .4137931 11544.824000000017 5 5
         4  1 1 .17261904 .26785713  .1904762  .10119048 .10119048 .065476194 .071428575 .029761905  .4404762 11544.824000000019 5 5
         5  1 1 .15384616  .3076923  .2167832  .07692308  .0909091 .062937066 .062937066  .02797203  .4755245 11550.838999999978 5 5
         6  1 1 .15441176  .2132353  .1617647  .13970588 .10294118  .11029412  .08088236 .036764707  .4191177 11550.838999999982 5 5
         7  1 1  .1769231  .2846154 .15384616   .1076923 .07692308  .07692308  .06153846  .06153846  .3615385 11550.838999999984 5 5
         8  1 1 .22292994  .2611465 .12738854  .10191083 .15286624  .06369427  .04458599  .02547771  .4585987 11550.838999999973 5 5
         9  1 1 .23776224  .2027972 .12587413   .0979021 .08391608  .13286713  .04195804  .07692308  .4825175  11570.02200000004 5 5
        10  1 1 .16197184 .26760563 .13380282  .14788732 .08450704   .0915493  .07042254  .04225352 .41549295 11570.022000000039 5 5
        11  1 1 .20437956 .23357664 .15328467  .13138686 .08029197  .08029197   .0729927  .04379562 .50364965 11570.022000000037 5 5
        12  1 1  .1983471  .2479339  .1818182  .14876033 .07438017   .0661157  .03305785  .04958678  .4214876 11570.022000000026 5 5
        13  1 1 .15686275  .2254902 .14705883  .19607843 .04901961  .05882353  .10784314  .05882353  .3921569 11594.408000000029 5 5
        14  1 1  .1567164 .23880596 .17910448  .09701493  .1641791  .08208955  .04477612  .03731343  .4253731 11594.408000000034 5 5
        15  1 1  .1551724  .1637931 .18103448   .1637931 .11206897   .0775862  .12068965  .02586207  .4396552 11594.408000000032 5 5
        16  1 1 .14864865  .2837838 .08108108   .1081081  .1081081   .0945946   .0945946  .08108108 .41216215 11594.408000000036 5 5
        17  1 1  .2184874  .1764706 .14285715  .05042017 .11764706  .16806723 .067226894  .05882353  .4957983 11605.090000000011 5 5
        18  1 1  .1743119  .1743119 .13761468   .0825688 .11009175  .11926606  .12844037  .07339449 .42201835 11605.090000000006 5 5
        19  1 1 .12903225 .20430107   .172043   .0967742 .13978495  .04301075  .11827957   .0967742  .4408602 11605.089999999991 5 5
        20  1 1 .14049587  .2231405  .2231405   .1322314 .05785124  .09917355  .09917355  .02479339   .446281 11605.090000000013 5 5
        21  1 1 .11206897 .20689656  .1637931  .11206897 .11206897  .10344828  .12068965  .06896552  .4913793  11614.37299999999 5 5
        22  1 1 .11607143  .2142857 .14285715  .14285715 .11607143 .071428575   .1517857  .04464286  .4196429 11614.372999999994 5 5
        23  1 1  .1825397  .1904762 .16666667  .15873016 .07936508   .1031746   .1031746 .015873017  .4365079 11614.372999999985 5 5
        24  1 1 .20212767 .11702128 .19148937  .14893617 .13829787  .08510638  .05319149  .06382979  .4574468 11614.373000000012 5 5
        25  1 1      .125 .23958333    .15625  .11458334 .10416666       .125     .09375  .04166667        .5  11658.60900000002 5 5
        26  1 1 .11864407  .2457627 .18644068  .11864407 .08474576  .09322034  .11864407 .033898305  .4745763 11658.609000000006 5 5
         1 16 1 .14492753 .28985506  .2463768 .072463766 .11594203  .10144927 .014492754 .014492754  .3623188 3065.2229999999986 0 5
         2 16 1  .1147541 .32786885 .14754099  .09836066  .1147541 .032786883   .1147541  .04918033  .3606557 3065.2229999999986 0 5
         3 16 1 .09433962 .26415095 .13207547   .1509434  .2264151   .0754717 .018867925  .03773585 .41509435 3065.2229999999986 0 5
         4 16 1  .1794872  .3333333 .12820514  .07692308  .1025641  .05128205  .05128205  .07692308  .4615385 3065.2229999999986 0 5
         5 16 1 .11627907 .25581396 .11627907   .1860465 .09302326  .09302326  .09302326  .04651163  .3255814  3076.309999999998 0 5
         6 16 1 .13725491 .15686275 .19607843   .1764706 .07843138   .1764706  .03921569  .03921569  .4901961  3076.309999999998 0 5
         7 16 1 .15686275 .21568628 .15686275  .19607843 .07843138  .07843138  .09803922 .019607844  .4705882  3076.309999999998 0 5
         8 16 1 .04444445  .1777778 .24444444  .15555556 .11111111  .13333334  .13333334          0  .3555556  3076.309999999998 0 5
         9 16 1 .12765957 .17021276 .25531915  .10638298 .17021276  .10638298 .021276595  .04255319  .3829787  3091.929999999996 0 5
        10 16 1 .27450982 .15686275 .15686275  .19607843 .05882353  .03921569  .09803922 .019607844 .43137255 3091.9299999999957 0 5
        11 16 1        .2  .3111111 .11111111  .13333334 .06666667  .13333334  .02222222  .02222222  .4888889  3091.929999999996 0 5
        12 16 1 .17307693 .23076923 .21153846  .11538462 .07692308  .09615385  .05769231  .03846154  .4423077 3091.9299999999957 0 5
        13 16 1       .16       .22       .16         .2        .1        .06        .08        .02       .34 3108.0299999999993 0 5
        14 16 1  .1851852  .0925926 .24074075  .16666667 .14814815   .0925926  .05555556 .018518519  .3333333 3108.0299999999993 0 5
        15 16 1 .08510638 .25531915 .19148937  .14893617 .10638298  .12765957 .021276595  .06382979  .5744681 3108.0299999999993 0 5
        16 16 1 .09433962 .18867925 .16981132  .09433962  .1509434  .09433962  .16981132  .03773585  .4716981 3108.0299999999993 0 5
        17 16 1       .04       .22       .22        .22        .1        .08        .06        .06        .3 3121.9970000000017 0 5
        18 16 1 .13636364       .25 .13636364   .0909091 .13636364   .0909091  .13636364 .022727273  .3863636 3121.9970000000017 0 5
        19 16 1     .0625     .1875 .29166666  .16666667 .14583333  .04166667  .04166667      .0625     .4375 3121.9970000000017 0 5
        20 16 1        .1       .26       .14         .1       .16        .08        .14        .02       .38 3121.9970000000017 0 5
        21 16 1 .13157895  .2631579  .2368421  .10526316 .02631579  .15789473  .07894737          0 .28947368 3134.6929999999998 0 5
        22 16 1 .16666667  .2037037  .2037037  .12962963 .16666667  .05555556  .05555556 .018518519 .44444445 3134.6929999999998 0 5
        23 16 1 .19565217  .2173913 .19565217  .13043478 .06521739  .08695652  .08695652  .02173913 .45652175 3134.6929999999998 0 5
        24 16 1        .1        .3      .125       .125      .125       .175          0        .05       .55 3134.6929999999998 0 5
        25 16 1        .2       .28       .12        .08       .08        .12        .12          0        .4 3145.7110000000002 0 5
        26 16 1  .0882353  .2352941 .14705883   .0882353  .1764706  .05882353   .1764706 .029411765  .3529412 3145.7109999999993 0 5
         1 17 1 .16666667 .16666667 .16666667   .1764706 .13725491  .09803922 .068627454 .019607844  .3333333 6822.5200000000095 0 5
         2 17 1       .17       .18       .17        .15       .08        .12        .09        .04       .43 6822.5200000000095 0 5
         3 17 1 .12820514 .16666667  .2179487  .11538462 .12820514  .07692308   .0897436  .07692308  .3589744  6822.520000000007 0 5
         4 17 1  .1011236 .19101124  .1797753  .08988764 .16853933   .1011236   .1235955  .04494382  .4606742  6822.520000000008 0 5
         5 17 1 .12162162 .16216215 .24324325  .16216215 .08108108  .12162162  .04054054 .067567565  .3648649   6895.22600000001 0 5
         6 17 1 .11627907 .12790698 .19767442  .13953489 .10465116   .1627907  .13953489 .011627907 .29069766  6895.226000000012 0 5
         7 17 1 .23376623 .25974026  .1948052 .064935066 .07792208  .11688311  .03896104 .012987013  .4675325  6895.226000000011 0 5
         8 17 1 .10465116 .23255815  .1511628  .12790698 .11627907  .09302326  .12790698  .04651163  .3372093  6895.226000000012 0 5
         9 17 1       .15       .23       .13        .12       .18         .1        .05        .04       .45  6968.006000000009 0 5
        10 17 1 .13414635  .2682927 .18292683  .04878049  .1097561  .13414635  .08536585 .036585364  .4268293  6968.005999999999 0 5
        11 17 1 .11764706 .10588235  .2235294  .07058824 .10588235   .2352941  .05882353  .08235294  .3647059  6968.006000000001 0 5
        12 17 1  .1612903  .2096774  .1935484   .1935484 .11290322 .016129032  .08064516 .032258064  .3387097  6968.005999999995 0 5
        13 17 1 .13483146  .2247191 .13483146   .1011236  .1573034  .06741573   .1235955  .05617978  .4382023  7054.195999999997 0 5
        14 17 1  .1369863 .24657534 .10958904  .10958904  .0958904  .10958904   .0958904   .0958904  .4383562  7054.195999999997 0 5
        15 17 1 .16666667 .17857143 .11904762  .14285715  .0952381  .14285715  .08333334 .071428575  .3452381  7054.195999999998 0 5
        16 17 1  .1547619 .17857143 .11904762  .16666667 .10714286  .17857143  .08333334 .011904762  .4404762  7054.195999999998 0 5
        17 17 1 .12222222  .1777778  .1888889  .16666667 .13333334   .0888889         .1  .02222222  .3666667            7160.29 0 5
        18 17 1 .16666667  .1547619 .14285715  .05952381  .1904762  .14285715  .10714286 .035714287  .3452381  7160.289999999997 0 5
        19 17 1  .1857143        .2 .12857144  .11428571 .08571429  .15714286  .11428571 .014285714  .3857143  7160.289999999992 0 5
        20 17 1 .20454545 .17045455 .14772727  .14772727  .0909091   .0909091  .07954545  .06818182  .4318182  7160.289999999999 0 5
        21 17 1 .17567568  .1891892  .2162162  .12162162 .02702703  .16216215  .08108108  .02702703   .527027               7288 0 5
        22 17 1   .203125     .1875    .09375      .0625    .09375    .078125    .140625    .140625   .390625               7288 0 5
        23 17 1  .1612903  .0967742 .11290322   .1935484  .1612903  .08064516  .12903225  .06451613  .4032258               7288 0 5
        24 17 1 .15384616 .23076923        .2  .04615385 .12307692  .06153846  .15384616  .03076923  .4153846               7288 0 5
        25 17 1  .1038961 .11688311 .15584415   .0909091 .14285715  .16883117  .15584415 .064935066  .4675325  7405.743000000006 0 5
        26 17 1 .20338982 .16949153 .08474576  .13559322 .10169491  .16949153  .10169491 .033898305  .2881356  7405.743000000003 0 5
         1 18 1 .08333334       .25       .25  .16666667 .08333334  .08333334          0  .08333334        .5  685.4759999999998 0 5
         2 18 1  .0909091  .0909091 .27272728   .0909091  .1818182   .0909091          0   .1818182  .4545455  685.4759999999998 0 5
         3 18 1 .11111111 .44444445 .22222222          0 .11111111  .11111111          0          0  .5555556  685.4759999999999 0 5
         4 18 1 .27272728  .0909091 .27272728   .0909091 .27272728          0          0          0  .3636364  685.4759999999998 0 5
         5 18 1       .15       .15        .4        .15         0          0          0        .15        .4  702.0869999999998 0 5
         6 18 1        .6        .2         0         .2         0          0          0          0        .2            702.087 0 5
         7 18 1      .125       .25         0       .375      .125          0       .125          0      .375            702.087 0 5
         8 18 1 .11764706 .29411766 .11764706          0  .1764706  .11764706  .11764706  .05882353 .29411766  702.0869999999998 0 5
         9 18 1  .0909091         0  .1818182  .27272728  .0909091   .1818182          0   .1818182 .54545456  724.0190000000001 0 5
        10 18 1         0        .1        .4         .1        .2          0         .1         .1        .2  724.0190000000001 0 5
        11 18 1 .14285715 .14285715 .14285715          0  .4285714  .14285715          0          0  .5714286  724.0190000000001 0 5
        12 18 1      .125      .375      .125       .125      .125       .125          0          0       .25  724.0190000000001 0 5
        13 18 1 .08333334 .16666667 .16666667  .16666667 .08333334  .16666667          0  .16666667  .8333333  739.9040000000001 0 5
        14 18 1        .1        .2         0          0        .4         .1         .2          0        .2  739.9040000000001 0 5
        15 18 1 .15384616 .15384616 .15384616   .3076923 .23076923          0          0          0  .4615385  739.9040000000002 0 5
        16 18 1 .05263158 .15789473  .3157895  .10526316  .2105263  .05263158  .05263158  .05263158  .4210526  739.9040000000002 0 5
        17 18 1 .14285715         0  .8571429          0         0          0          0          0  .5714286            756.835 0 5
        18 18 1         0 .16666667         0   .3333333 .16666667          0  .16666667  .16666667  .8333333            756.835 0 5
        19 18 1         0  .4615385         0          0  .3846154  .07692308  .07692308          0  .3846154  756.8349999999999 0 5
        20 18 1 .14285715  .4285714  .2857143  .14285715         0          0          0          0  .2857143            756.835 0 5
        21 18 1        .2        .1        .1          0        .1         .3         .2          0        .3  757.9520000000001 0 5
        22 18 1         0  .6666667         0  .16666667         0          0  .16666667          0         0  757.9520000000001 0 5
        end
        
        local outcome "narc"
        local controlvars "age015 age1624 age2534 age3544 age4554 age5564 age65plus agemiss male pop"
        collapse (mean) `outcome' `controlvars' qtrtreat, by (trunit qtr)
        // keep if trunit==1|trunit>15
        egen maxp_1=max(qtrtreat)
        local p=maxp_1
        display `p'
        local p1=`p'-1
        display `p1'
        
        xtset trunit qtr
        
        keep if inrange(qtr,1,22) // need to have a balanced panel for synth to work
        
        synth `outcome' `controlvars', trunit(1) trperiod(5) fig keep(synthetic_casestudy_narc_1, replace) 
        synth `outcome' `controlvars' narc(1) narc(2) narc(3) narc(4), trunit(1) trperiod(5) fig keep(synthetic_casestudy_narc_1, replace)

        Comment


        • #5
          You are the best! Thank you so much. I feel pretty dumb about the mistake I made and it totally baffled me. I got it now and thank you for reminding me to re-check my data evertime after (mindlessly) moving code around.

          I have made some progress; thank you! If I may, I have another question now. I totally understand if you cannot answer it and I need to start another thread. But here it is in case you are able to help:

          As I had mentioned, I have multiple (15) treated states, being treated at different times. So I eventually want to put the synth inside a loop to do 15 case studies. My loop looks like the following:

          Code:
          tsset trunit qtr
          forvalues lname = 1/15 {
          preserve
          keep if (trunit==`lname'|trunit>15)
          egen maxp_`lname'=max(qtrtreat)
          local p=maxp_`lname'
          display `p'
          local p1 = `p' - 1
          display `p1'
          /*qui*/ synth narc `controlvars' narc(1(1)`p1') , trunit(`lname') trperiod(`p') fig keep(synthetic_casestudy_narc_`lname', replace)    
          restore
          }
          So the number of lags, of the outcome narc, is dynamic. But, what the code, as I have written it above, is simply averaging over the periods 1-`p1' and including that average as a predictor. But that is not what I really want(should) to do. What I need is to include all the individual lags: narc(1) narc(2) narc(3)...narc(`p1'-1) narc(`p1'). How would I code that? Sorry for all the questions... but I am very grateful for your help.
          Sincerely,
          Sumedha.

          Comment


          • #6
            You can do something like this, perhaps more elegantly:

            Code:
            cls
            /* Fake Data */
            clear
            set obs 5
            gen trunit = _n
            expand _n
            bys trunit: gen qtr = _n
            
            /* Synth loop */
            forvalues lname = 1/5 {
                
                quietly levelsof qtr if trunit==`lname', local(levels)
                
                quietly sum qtr if trunit==`lname'
                local last_period = r(max)
                local levels : list levels - last_period // remove the last period from the list 
            
                if "`levels'" != "" {
                    local levels = ustrregexra("`levels'","^","narc(")  // insert the first narc( to get narc(1    
                    local levels = ustrregexra("`levels'"," "," narc(") // insert all the intermediate narc(s
                    local levels = ustrregexra("`levels'"," ",") ")     // close the parentheses
                    local levels = ustrregexra("`levels'","$",")")        // insert the last parenthesis
                }
                display "Lags to include for unit `lname' are: `levels'"
            }
            Remove the quietly and add more displays to see what is happening under the hood.

            By the way, I am not a big fan of the

            Code:
             
             local p=maxp_`lname'
            This defines a local named p that contains the first element of max_`lname'. Since all the elements of that variable are presumably the same, this is OK in this case and doesn't depend on the sort order of the data. But in general, this seems like a bad habit to pick up.

            Comment


            • #7
              Adopting Robert Picard's suggestion from another thread, this could be done a bit more simply:

              Code:
              /* Fake Data */
              clear
              set obs 5
              gen trunit = _n
              expand _n
              bys trunit: gen qtr = _n
              
              /* Synth loop */
              forvalues lname = 1/5 {
                  
                  quietly levelsof qtr if trunit==`lname', local(levels)
                  
                  quietly sum qtr if trunit==`lname'
                  local last_period = r(max)
                  local levels : list levels - last_period // remove the last period from the list 
              
                  if "`levels'" != "" {
                      local levels = ustrregexra("narc("+"`levels'"+")"," ", `") narc("')    
                      display "Lags to include for unit `lname' are: `levels'"
                  }
              }

              Comment


              • #8
                Dear Prof. Masterov,

                Thank you so much for the code. This has been so useful and I am very grateful. I think I am still doing something incorrectly though.

                I have coded things to include as predictors in the synth a set of demographic variables (`controlvars') and lags of the outcome variable, using the code you just sent above. But when synth generates the synthetic controls it only seems to include as predictors the lags of the outcome variable and not the demogrphic variables for some reason. Here is the code and output for reference:
                Code:
                 
                local outcome narc otherreason misuse suicide reasonunknown nomineff modeff majeff death effmiss
                local controlvars  age015 age1624 age2534 age3544 age4554 age5564 age65plus agemiss male  
                
                bys trunit: gen lags = _n if qtr<=qtrtreat
                (679 missing values generated)
                
                .
                . tsset trunit qtr
                       panel variable:  trunit (strongly balanced)
                        time variable:  qtr, 1 to 26
                                delta:  1 unit
                
                . forvalues lname = 1/1 {
                  2. preserve
                  3. keep if (trunit==`lname'|trunit>15)
                  4. egen maxp_`lname'=max(qtrtreat)
                  5. local p=maxp_`lname'
                  6. display `p'
                  7. local p1 = `p' - 1
                  8. display `p1'
                  9. quietly levelsof lags if trunit==`lname', local(levels)
                 10. quietly sum lags if trunit==`lname'
                 11.     local last_period = r(max)
                 12.     local levels : list levels - last_period // remove the last period from the list
                >  
                 13.
                .     if "`levels'" != "" {
                 14.         local levels = ustrregexra("narc("+"`levels'"+")"," ", `") narc("')    
                 15.         display "Lags to include for unit `lname' are: `levels'"
                 16.     }
                 17. /* synth narc `controlvars' narc(1(1)`p1') , trunit(`lname') trperiod(`p') fig keep(
                > synthetic_casestudy_narc_`lname', replace)  */
                . synth narc `controlvars' `levels' , trunit(`lname') trperiod(`p') fig keep(synthetic_ca
                > sestudy_narc_`lname', replace)
                 18. restore
                 19. }
                (364 observations deleted)
                5
                4
                Lags to include for unit 1 are: narc(1) narc(2) narc(3) narc(4)
                -----------------------------------------------------------------------------------------
                Synthetic Control Method for Comparative Case Studies
                -----------------------------------------------------------------------------------------
                
                First Step: Data Setup
                -----------------------------------------------------------------------------------------
                -----------------------------------------------------------------------------------------
                Data Setup successful
                -----------------------------------------------------------------------------------------
                                Treated Unit: 1
                               Control Units:  16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
                -----------------------------------------------------------------------------------------
                          Dependent Variable: narc
                  MSPE minimized for periods: 1 2 3 4
                Results obtained for periods: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
                                              24 25 26
                -----------------------------------------------------------------------------------------
                                  Predictors: narc(1) narc(2) narc(3) narc(4)
                -----------------------------------------------------------------------------------------
                Unless period is specified
                predictors are averaged over: 1 2 3 4
                -----------------------------------------------------------------------------------------
                
                Second Step: Run Optimization
                -----------------------------------------------------------------------------------------
                -----------------------------------------------------------------------------------------
                Optimization done
                -----------------------------------------------------------------------------------------
                
                Third Step: Obtain Results
                -----------------------------------------------------------------------------------------
                Loss: Root Mean Squared Prediction Error
                
                ---------------------
                   RMSPE |   .000574
                ---------------------
                -----------------------------------------------------------------------------------------
                Unit Weights:
                
                -----------------------
                    Co_No | Unit_Weight
                ----------+------------
                       16 |        .313
                       17 |           0
                       18 |        .295
                       19 |           0
                       20 |           0
                       21 |           0
                       22 |           0
                       23 |           0
                       24 |           0
                       25 |        .011
                       26 |           0
                       27 |           0
                       28 |        .381
                       29 |           0
                       30 |           0
                       31 |           0
                       32 |           0
                       33 |           0
                       34 |           0
                -----------------------
                -----------------------------------------------------------------------------------------
                Predictor Balance:
                
                ------------------------------------------------------
                                               |   Treated  Synthetic
                -------------------------------+----------------------
                                       narc(1) |   .016371   .0157561
                                       narc(2) |  .0129928   .0137258
                                       narc(3) |  .0125597   .0131195
                                       narc(4) |   .014552   .0142527
                ------------------------------------------------------
                -----------------------------------------------------------------------------------------
                
                .
                end of do-file
                
                .
                The first loop is there because eventually I want to run multiple case studies. Any thoughts on why the `controlvars' are being left out from the predictors list?

                Thank you again.
                Sincerely,
                Sumedha.

                Comment


                • #9
                  I have no idea what is going wrong. Why don't you insert:

                  Code:
                  display "Controls are `controlvars'";
                  before the synth step.

                  Comment

                  Working...
                  X