Difference-in-Differences and Unbalanced Samples: Is TWFE Biased?

[Martin Budger]

Thanks for the great explanation. I have a related question. Since we know that TWFE is typically biased in settings where the treatment occurs at different times, I am wondering about a case where the treatment effect occurs at a single time point (and persists thereafter), but the sample consists of three groups:

a) Units that are present both before and after the treatment date.
b) Units that exist only before the treatment.
c) Units that exist only after the treatment date.

Each of these groups contains both treated and control units. I am wondering whether applying the TWFE estimator to such a sample would be biased and whether one of the newer estimators (eg. C-S) should be used instead. Alternatively, when using TWFE, should groups (b) and (c) be dropped?

Put differently, should each unit in both the control and treatment groups be observed at least twice—once in the pre-treatment period and once in the post-treatment period? If so, should observations from units that are available only in the pre-treatment or post-treatment period be dropped?

I assume that before the rise in awareness of TWFE biases in settings with different treatment timing, using such a setup was considered acceptable. However, I am now wondering whether TWFE remains biased in this case due to weighting issues or other factors.

Would appreciate any insights or references on this issue!

[Seungmin Lee]

I think (b) and (c) should be dropped, not because of bias but because you simply can't estimate.

Basically DiD estimators are generated by comparing (pre- and post-), and (treatment and control). For (b) and (c), you don't have outcomes (post- is missing in (b), and pre- is missing in (c)) to compute estimators. Even if you don't drop them, I assume they will be automatically dropped from your regression sample due to the absence of data.

If it's about those who exist both in pre- and post- treatment date, but don't stay in the entire study period (joined later, or dropped earlier), I can't give you the clear answer. The latest DnD working paper (here) says it doesn't matter but the interpretation of the estimator might change, so you might wanna check it from your own end.

[Martin Budger]

thank you Seungmin!