Notation to include outcome variables in the previous perioid as controls.

[Sumedha Gupta]

Dear All,

I am trying to estimate synthetic control difference-in-difference case studies using the synth package introduced in
Abadie, A., Diamond, A., and J. Hainmueller. 2010. Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program. Journal of the American Statistical Association 105(490): 493-505. (http://fmwww.bc.edu/RePEc/bocode/s/synth.html).

I have state-quarter data from 34 states. 15 states implement a policy change, but at different times and 19 states serve as the donor pool to create the synthetic controls. As is standard, the outcome variable in the quarters prior to the treatment perioid are included as explanatory variables in the synth estimation. Since, each of the 15 states implements the policy change in different periods the number of lagged outcome variables to be included as explanatory variables differs in each case study.

I have written the following to do the 15 different case-studies...but I don't know how to incorporate the appropriate lagged outcome variables as controls:

local outcome narc
local controlvars age015 age1624 age2534 age3544 age4554 age5564 age65plus agemiss male pop

tsset trunit qtr

forvalues lname = 1/15 {
preserve
local t=`lname'
display `t'
keep if (trunit==`lname'|trunit>15)
egen maxp_`lname'=max(qtrtreat)
local p=maxp_`lname'
display `p'
qui synth `outcome' `controlvars' `outcome'(1(1)(`p'-1)) , trunit(`t') trperiod(`p') fig keep(synthetic-casestudy`lname', replace)
restore
}

Stata gives me the error: (1(1)(5-1)) does not exist as a (numeric) variable in dataset r(198); I think I am not including the lags correctly. I will appreciate any help I can get please. Sincerely, Sumedha.

[William Lisowski]

Let's start by using CODE delimiters to render your code readably.

ocal outcome narc
local controlvars age015 age1624 age2534 age3544 age4554 age5564 age65plus agemiss male pop

tsset trunit qtr

forvalues lname = 1/15 {
   preserve
   local t=`lname'
   display `t'
   keep if (trunit==`lname'|trunit>15)
   egen maxp_`lname'=max(qtrtreat)
   local p=maxp_`lname'
   display `p'
   qui synth `outcome' `controlvars' `outcome'(1(1)(`p'-1)) , trunit(`t') trperiod(`p') fig keep(synthetic-casestudy`lname', replace)  
   restore
}

This

`outcome'(1(1)(`p'-1))

is perhaps the cause of your error message, because the output of help numlist does not include constructs like, for example, 1(1)(5-1) which is what your code becomes after substituting 5 for `p'. There are several ways to correct this, the most obvious is

local p1 = `p' - 1
display `p1'
qui synth `outcome' `controlvars' `outcome'(1(1)`p1') , trunit(`t') trperiod(`p') fig keep(synthetic-casestudy`lname', replace)

Without knowing more about synth than reading far enough into the output of help synth (a user-written extension to Base Stata, from SSC) to understand the command syntax, I cannot assess whether this will wholly solve your problems. But at least you won't get the error message you reported in post #1.

To present data, code, and results readably, please copy them from the Results window or elsewhere and paste them into a code block in the Forum editor, as explained in the Statalist FAQ linked to at the top of the page. For example, the following:

[CODE]
// sample code
sysuse auto, clear
describe
[/CODE]

will be presented in the post as the following:

// sample code
sysuse auto, clear
describe

[Sumedha Gupta]

Thank you Prof. Lisowski! Your code helped me solve the error I was getting. Now, I have encountered the next error. I changed my code to capture what you mentioned

local outcome narc otherreason misuse suicide reasonunknown nomineff modeff majeff death effmiss
local controlvars  age015 age1624 age2534 age3544 age4554 age5564 age65plus agemiss male pop

collapse  (mean) `outcome' `controlvars' qtrtreat , by (trunit qtr)

tsset trunit qtr

foreach y in `outcome' {
forvalues lname = 1/15 {
preserve
keep if (trunit==`lname'|trunit>15)
egen maxp_`lname'=max(qtrtreat)
local p=maxp_`lname'
display `p'
local p1 = `p' - 1
display `p1'
qui synth `y' `controlvars' `y'(1(1)`p1') , trunit(`lname') trperiod(`p') fig keep(synthetic_casestudy_`y'_`lname', replace)    
restore
}
}

It actually runs but gives me no weights nor synthetic predticted y's. For instance, for the first outcome 'narc' and trunit==1 I get the following Stata data file:

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(_Co_Number _W_Weight) double(_Y_treated _Y_synthetic _time)
16 . 1 .  1
17 . 1 .  2
18 . 1 .  3
19 . 1 .  4
20 . 1 .  5
21 . 1 .  6
22 . 1 .  7
23 . 1 .  8
24 . 1 .  9
25 . 1 . 10
26 . 1 . 11
27 . 1 . 12
28 . 1 . 13
29 . 1 . 14
30 . 1 . 15
31 . 1 . 16
32 . 1 . 17
33 . 1 . 18
34 . 1 . 19
 . . 1 . 20
 . . 1 . 21
 . . 1 . 22
 . . 1 . 23
 . . 1 . 24
 . . 1 . 25
 . . 1 . 26
end

This is obviously not correct. I think I am making some sort of loop error but am not sure what. I will be grateful for any help that could be offered.
Sincerely,
Sumedha.

[William Lisowski]

My guess is that the synth command is failing, but because you have used the quiet prefix before the command, you are not seeing the diagnostic messages it would produce to tell you so. You should never use the quiet prefix until you know your code is working.

To test your code, I would do something like the following.

local outcome narc otherreason misuse suicide reasonunknown nomineff modeff majeff death effmiss
local outcome narc // test with one outcome, fix the problems, add a second outcome and test again, then remove this command 
local controlvars  age015 age1624 age2534 age3544 age4554 age5564 age65plus agemiss male pop
...
// comment the quiet out for testing, remove the /* and */ when testing is complete
/* quiet */ synth `y' `controlvars' `y'(1(1)`p1') , trunit(`lname') trperiod(`p') fig keep(synthetic_casestudy_`y'_`lname', replace)
...

[Sumedha Gupta]

Thank you, Prof. Lisowski. I followed your suggestions and got the following:

local outcome narc /*otherreason misuse suicide reasonunknown nomineff modeff majeff de
> ath effmiss */

. local controlvars  age015 age1624 age2534 age3544 age4554 age5564 age65plus agemiss mal
> e pop

.
. collapse  (mean) `outcome' `controlvars' qtrtreat , by (trunit qtr)

.
. tsset trunit qtr
panel variable:  trunit (strongly balanced)
time variable:  qtr, 1 to 26
delta:  1 unit

.
. foreach y in `outcome' {
2. forvalues lname = 1/15 {
3. preserve
4. keep if (trunit==`lname'trunit>15)
5. egen maxp_`lname'=max(qtrtreat)
6. local p=maxp_`lname'
7. display `p'
8. local p1 = `p' - 1
9. display `p1'
10. /*qui*/ synth `y' `controlvars' `y'(1(1)`p1') , trunit(`lname') trperiod(`p') fig ke
> ep(synthetic_casestudy_`y'_`lname', replace)       
11. restore
12. }
13. }
(364 observations deleted)
5
4

Synthetic Control Method for Comparative Case Studies


First Step: Data Setup


Data Setup successful

Treated Unit: 1
Control Units:  16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

Dependent Variable: narc
MSPE minimized for periods: 1 2 3 4
Results obtained for periods: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
24 25 26

Predictors: age015 age1624 age2534 age3544 age4554 age5564 age65plus
agemiss male pop narc(1(1)4)

Unless period is specified
predictors are averaged over: 1 2 3 4


Second Step: Run Optimization


Optimization done


Third Step: Obtain Results

Loss: Root Mean Squared Prediction Error


RMSPE          .


Unit Weights:


Co_No  Unit_Weight

16            
17            
18            
19            
20            
21            
22            
23            
24            
25            
26            
27            
28            
29            
30            
31            
32            
33            
34            


Predictor Balance:


Treated  Synthetic

age015   .1887048          .
age1624   .2452776          .
age2534    .183628          .
age3544   .1065218          .
age4554   .0979623          .
age5564   .0652333          .
age65plus   .0674047          .
agemiss   .0452675          .
male   .4161599          .
pop   11544.82          .
narc(1(1)4)          1          .


(364 observations deleted)

So the error is in

Root Mean Squared Prediction. But, I am not sure what exactly happens... Also why it simply includes narc(1(1)4) instead of narc(1) , narc(2), narc(3) and narc(4) in the predictors.

I will be grateful for any help.
Sincerely,
Sumedha.

[William Lisowski]

At this point, I've done as much as I can, since I don't know anything substantial about synth. I will say that it looks like your output is consistent with the printed results; the problem is that synth seems to not be doing what you expected. I will say that seeing

Loss: Root Mean Squared Prediction Error


RMSPE          .

is very suspicious to me: it tells you the loss function is root mean squared prediction error, then it displays a missing value for it. To me, it is suggestive of your model not quite meeting the assumptions of synthetic estimation. Perhaps someone else will have an idea.

if you don't get any answers in a day or so, I'd suggest starting a new topic with a title that better describes the current problem - something like "synth command is producing missing values for its results" - then show the results as you did in post #5. Right now, the members who know synth have no reason to read this post - they don't know their knowledge might help.

Let me add a hints about posting code: don't try to copy from your post above to a new post, it may work but it often has glitches. Instead, rerun the code and make a new copy-and-paste.