Small not-yet-treated groups in csdid with staggered adoption

[Yuzhang Zhang]

Hi everyone,

I am using the Callaway & Sant’Anna (2021) DID estimator (csdid) in a staggered adoption setting. I only have not-yet-treated as my control group. After examining the (g,t) cells, I noticed that for some cohorts, particularly later adopters, the number of not-yet-treated units becomes very small (sometimes only 1–3 observations).

My ATT estimate is unusually large and unstable, which makes me concerned about weak support.

My questions are:

1. When the not-yet-treated comparison group is extremely small, should those (g,t) cells be considered unreliable?
2. Is it acceptable practice to trim cells based on a minimum control group size (e.g., requiring at least 5–10 controls)?
3. Is csdid still appropriate, or should alternative approaches such as Synthetic DID be considered under this occasion?

Any guidance on best practices would be greatly appreciated.

Many thanks.

[FernandoRios]

Hi
you are on point. if you comparison group is small, your results will be very volatile. so you could restrict your sample to avoid certain groups (which requires some care when making the specification)
Based on your description, you are right that other approaches may be more approrpiate where either you use a larger pool of comparison groups, of fewer control variables, or stronger assumptions.
F

[Jeff Wooldridge]

One possibility is to use regression adjustment and exact inference after collapsing the data to cross-sectional data. My coauthor, Soo Jeong Lee, and I show how to do this in the paper here.

All that is required is subtracting off pre-treatment averages (or trends), unit-by-unit. Then, apply linear regression. It can work even with N0 = 2 (control units) and N1 = 1 (treated units). And it works for any number of time periods -- large or small. Synthetic DiD is another option, but it can require many units in the donor pool, which you don't always have.

You have to be very wary about including controls. The Lee and Wooldridge paper shows you run out of degrees-of-freedom quickly if you try the fully flexible approach in Callaway-Sant'Anna.