difference-in-difference with single treated unit

[Liz Gooch]

Hi - I am investigating a quantitative problem with one treated unit and a single time for the treatment onset and around 21 untreated units using the synthetic control method. I was wondering if someone could shed some light on whether there are any bias or inference issues with applying a difference-in-difference methodology to this situation? I am particularly interested in knowing if methodological problems/challenges may arise from the singularity of the treated unit.

Appreciatively,
Liz

[George Ford]

synthetic is the way to go. lots of issues with a single treated unit in more traditional regression based approaches.

[Jeff Wooldridge]

Unfortunately, the validity of SC requires many post-treatment periods -- otherwise the inference is not valid. I've proposed a different approach based on the representation of DiD as a cross-sectional regression. You average the pre-treatment periods and subtract, for each unit (treated or otherwise), from the average of the post-treatment period. In this case, the single time period. Now you have a single cross section, and you can regress the differenced variable on a constant and the treatment dummy. If you believe the classical linear model assumptions hold for this cross-sectional equation, you can use the usual t statistic. With a single treated unit, this is identical to "outlier" analysis, which makes sense: you are testing whether the treated unit is an outlier compare with the controls. The coefficient on the treatment dummy is the Studentized residual from outlier analysis. I discuss this in Section 9.5 of my introductory econometrics book. In case it is useful, I'm attaching slides from a workshop I've taught. In Section 5, I show how to collapse the data and do the simple OLS regression. With 21 control units, you could even include some covariates. I have the Stata code, too, but it's pretty simple.

[George Ford]

I was thinking "single time for the treatment onset" meant a non-staggered treatment.

Looking forward to reviewing the slides.

.

[Jeff Wooldridge]

Definitely not staggered. Things are even easier in the common timing case, which is Liz’s setting.

[George Ford]

Liz, to clear things up, does "single time for the treatment onset" mean 1 period during treated period or just one treatment?

[Jeff Wooldridge]

Ah, I see how one could interpret what Liz described as common timing with many treated periods and not just a single treated period. The method I described still works, but SDID also would work unless the number of treated periods is small.

[Liz Gooch]

Thank you, George and Jeff for addressing my question.

To clear things up - I do mean common timing in the way Jeff interpreted it. Such that the one treated unit is untreated for a number of periods and then becomes treated for the rest of the time periods.

I have a few questions regarding your comments (again, I really appreciate your help!).

1. George, would you be willing to specify the issues with a single treated unit with regards to a traditional regression based method? I have the intuition that there are issues but am struggling to find evidence in the literature that lays out the problems, specifically.
2. Jeff, would you please provide me with an idea of what is "enough" treated time periods for SCM?

I am pretty excited to try to implement the methodology that describe, Jeff.

[Andrew Musau]

[*]George, would you be willing to specify the issues with a single treated unit with regards to a traditional regression based method? I have the intuition that there are issues but am struggling to find evidence in the literature that lays out the problems, specifically.

Have a look at Conley and Taber (2011), where this is discussed (see excerpt below):

Reference:

Conley, Timothy G., and Christopher R. Taber. 2011. “Inference with 'Differences in Differences' with a Small Number of Policy Changes,” The Review of Economics and Statistics, 93(1), pp. 113-125.

[Carlo Lazzaro]

Liz:
in addition to Jeff's enlightening slides, you may want to take a look to the following papers:

1. Abadie A, Gardeazabal J. The economic costs of conflict: a case study of the Basque country. Am Econ Rev. 2003;93(1):113-132. doi: 10.1257/000282803321455188.
2. Abadie A, Diamond A, Hainmueller J. Synthetic control methods for comparative case studies: estimating the effect of California’s tobacco control program. J Am Stat Assoc. 2010;105(490):493-505. doi: 10.1198/jasa.2009.ap08746.
3. Abadie A, Diamond A, Hainmueller J. Comparative politics and the synthetic control method. Am J Pol Sci. 2015;59(2);495–510. doi: 10.1111/ajps.12116.
4. Abadie A. Using synthetic controls: feasibility, data requirements, and methodological aspects. J Econ Lit. 2021, 59(2), 391–425. doi: 10.1257/jel.20191450.

The one in bold is the one I have learnt the most about SCM.

In addition, another must read is:

1. Athey S, Imbens GW. The state of applied econometrics: causality and policy evaluation. J Econ Perspect. 2017;31(2):3–32. doi: 10.1257/jep.31.2.3.

[Liz Gooch]

Thank you for helpful responses, Andrew and Carlo.

Andrew, am I correct in concluding that if the linear trends assumption is valid for a single unit DID (which is just a likely as with a larger number of treated units), then the only issue to be aware of is for inference?

[George Ford]

Inference is the biggy. There are several options--bootstrap, randomized inference, Synthetic Counterfactuals, etc. One advantage of SC is that it is now widely accepted and you don't have to convince your referees, who may not know anything about these problems, that your randomized inference solution is correct. In fact, you don't have to acknowledge any problem at all.

I've had good luck with -fect- for these sorts of problems.

[Liz Gooch]

Thank you all! This conversation has been very informative.

[Liz Gooch]

Unfortunately, the validity of SC requires many post-treatment periods -- otherwise the inference is not valid. I've proposed a different approach based on the representation of DiD as a cross-sectional regression. You average the pre-treatment periods and subtract, for each unit (treated or otherwise), from the average of the post-treatment period. In this case, the single time period. Now you have a single cross section, and you can regress the differenced variable on a constant and the treatment dummy. If you believe the classical linear model assumptions hold for this cross-sectional equation, you can use the usual t statistic. With a single treated unit, this is identical to "outlier" analysis, which makes sense: you are testing whether the treated unit is an outlier compare with the controls. The coefficient on the treatment dummy is the Studentized residual from outlier analysis. I discuss this in Section 9.5 of my introductory econometrics book. In case it is useful, I'm attaching slides from a workshop I've taught. In Section 5, I show how to collapse the data and do the simple OLS regression. With 21 control units, you could even include some covariates. I have the Stata code, too, but it's pretty simple.

Jeff, would it be possible to access your Stata code? Thank you again for your input.

[Jeff Wooldridge]

Liz: Sorry for the long delay. If you see this and send an email then I'll send the Stata code.