Synthetic control: 2 out of 5 donor pool units recieve 0 weight in all models

[Emil Alnor]

Hi all

I am using the synthetic control method to estimate the effects of a university teaching reform (which happened in 2008) on the wage level of its graduates. As control variables I use the high-school GPA average of the graduates, proportion of males, proportion of non-westerns and proportion where parents have at least short-cycle higher education. I have 1 treated university and 5 non-treated universities. I use the -synth- command from SSC. No matter if I look at the wage 1, 2, 3 or 4 years after graduation or if I set the treatment year to be 2008, 2009, 2010, 2011 and 2012, the same two universities recieve a weight of 0, while the other three universities recive appr. 25%, 25% and 50%.

It seems a little odd that this is the case across all my models, and that they recieve exactly 0 weight, instead of, say, 0.02 in some models and 0,023 in other model. But maybe this is just a general thing with synthetic control? I can't provide the data I am using as this is confidential, but I am hoping that someone can provide me with some generic advice on where to look for errors, when donor pool units recieve 0 weight.

[Victor Ellenbroek]

You may be interested in this recent article,

Arkhangelsky, Dmitry, Susan Athey, David A. Hirshberg, Guido W. Imbens, and Stefan Wager (2021). ‘Synthetic Difference in Differences’, American Economic Review, 111:12, 4088–4118 http://arxiv.org/abs/1812.09970.

which discusses some pitfalls with DID and SC (inclusing that synthetic control tends to weigh one or more control units more heavily than others, which is potentially problematic, see page 4094). It proposes a synthetic did design that uses unit and time weights.

sdid package:

Pailañir, Daniel, and Damian Clarke (2022). ‘SDID: Stata module to perform synthetic difference-in-differences estimation, inference, and visualization.’, https://github.com/Daniel-Pailanir/sdid.

[Jared Greathouse]

The problem here is that your weights are not sparse. Or, more precisely, that the target unit falls outside of the convex hull of the donor pool units. Ideally for synth you want your weights to be the reverse, i.e., 2 weights greater than 0 and 3 that equal 0. See my under review paper for more on this point.


In all seriousness, you have only 5 donors............. not illegal, but maybe expecting a sparse solution here is a fool's errand (it could be, I'm not saying it is). If I were you, I'd use scul (my estimator on ssc) and use a ridge or elastic net penalty.

[Emil Alnor]

Thanks both, I will look into your proposed solutions.

[Jared Greathouse]

Can you collect more donors? Or do you only have data on 6 schools

[Emil Alnor]

Can you collect more donors? Or do you only have data on 6 schools

I only have data on 6 schools.

[Jared Greathouse]

Yeah then I would likely go for Ridge or Elastic-Net here. Likely doesn't make sense to want a sparse solution.

[Emil Alnor]

Yeah then I would likely go for Ridge or Elastic-Net here. Likely doesn't make sense to want a sparse solution.

Okay, thanks, I will look into that.