Assigning treatment and control groups for DID analysis

[Adarsh Raghunandanan]

Dear Statalist members,

I am working on a difference-in-differences (DID) analysis to evaluate the effect of a regulation that was intended to assist weak MSME firms. A key issue I face is how to define “weak firms” for treatment assignment.

One possible approach I am considering is to classify a firm as “weak” if it has had negative book value in a rolling 4-year window preceding the year throughout the sample period. That is, treatment membership is determined by whether the firm’s book value is negative for the preceding 4 years, such that treated firms (weak firms) would change on a yearly basis.

My questions are:

1. Can I define treatment membership this way (i.e., based on a rolling criterion rather than a single-period characteristic)?
2. Would such a definition create any problems for identification in DID, such as introducing selection bias or violating the parallel trends assumption?

Thank you.

[George Ford]

How is "weak" defined in the regulation?

[Adarsh Raghunandanan]

Thank you George for the reply. For simplicity, I am refering to a published paper, which has done a similar exercise. Here, they have defined a firm as distressed if the firm is having a ccumulated losses greater than or equal to 50% of the average networth in the preceding four years, throughout the sample period (Before and after the regulation).

Here, what I can understand from here is that the treated firms will change every year before and after the intervention. Is this okay?

Thanks.https://www.sciencedirect.com/science/article/pii/S0929119920302807?ref=pdf_download&fr=RR-2&rr=985908d3bc3c7f39

[George Ford]

I would think the dummy would be defined using data before the treatment is applied. If the treatment is changing over time, then you've a staggered DID, possibly with entry/exit, which is not what is being estimated. Tricky.

the 'immediately preceding four years" suggest to me Distress is based on pre-treatment data only.

[Clyde Schechter]

On the other hand, looking at the model equation quoted in #3, IBC is apparently the treatment-in-effect indicator as it is interacted with the Distress variable and it has only a t subscript. But the Distress variable has three subscripts, suggesting it varies over time within both i and n entities (whatever those might be). That, to me, suggests that it is re-calculated at each time period using a rolling window of four-years lagging. This also fits with the wording that Distress "if a firm in a year has accumulated losses..." [emphasis added].

[George Ford]

I think Clyde is right. It's a staggered model, so that 2x2 model is likely to produced biased estimates. 2021 is about the same time Goodman-Bacon came out, so the authors probably weren't aware of the issue. I suspect there are thousands of papers that suffer the same problem.

[Adarsh Raghunandanan]

Thank you for your responses George and Clyde. Just to clarify, n refers to the industry of the firm. I have one more query. Is it fine to change the membership of firms into distressed and non distressed based on the prior 4 year rolling window even after the treatment? Because I have seen papers making the classification before the treatment and keep the membership of firms unchanged after the treatment.

Thanks a lot.

[George Ford]

That's the rub. If you set the treated as Distressed before 2016, then you got a "single" treatment timing, yet if firms first qualify as distressed in 2017 can be treated, you've got treated units being used as controls. I don't know how you get away from a staggered/event study type approach here. I think jwdid could handle it in an OLS framework. I suppose you could drop any firm that qualifies as distressed after 2016, but need to think about the implications.

Question: Being distressed doesn't mean you're treated, but only that you could be treated?