Modertaion versus Subsample analysis - Difference-in-differences analysis

[Pranshu Tripathi]

Dear Members,

I am conducting a difference-in-differences analysis in a panel data setting using a bankruptcy law as policy and a group dummy (treated and control).
My model is

1. DID:

xtreg debt/equity i.group#i.post x1 x2 x3 x4 i.t##i.iid, fe vce (robust)

2. I also tried to check through subsample analysis, i.e.,

xtreg debt/equity group x1 x2 x3 x4 i.t##i.iid, fe vce (robust), if treated or control

The result is insignificant for DiD, but subsample analysis shows positive and significant for control and negative and significant for treated. How should I interpret it considering the impact of policy on the choice between debt and equity?

(What is difference between moderation and subsample analysis?)

[Clyde Schechter]

The finding that an explanatory variable is statistically significant in one group and not statistically significant in the other group does not tell you anything about the significance of the difference between the groups. If you want to assess the difference between groups through separate regressions, then you will need to do that with -suest-,which, unfortunately, will not work with -xtreg-, so you will have to emulate -xtreg- with -regress- instead. It will look something like this:

regress outcome i.post x1 x2 x3 x4 i.t##i.id if treated // N.B.  DO NOT USE -vce(robust)- HERE
estimates store treated_group
regress outcome i.post x1 x2 x3 x4 i.t##i.id if control // AGAIN, DO NOT USE -vce(robust)- HERE
estimates store control_group
suest treated_group control_group, vce(robust)

lincom _b[treated_group_mean:1.post] - _b[control_group_mean:1.post]

Now, in your situation, this will not produce the same results you got from using the interaction terms method. That's because you did not interact post with the x variables, so in that model you are implicitly constraining the coefficients of x1-x4 to be the same in the treatment and control groups, whereas the separate regressions model does not impose that constraint. That constraint, of course, also exerts consequences on the estimated effects of post, which is why these two methods will not produce identical results. An interaction term approach that will reproduce the results you get from separate regressions would be (assuming that your -xtset- statement declared id as the panel variable):

xtreg outcome i.treat##(i.post c.x1 c.x2 c.x3 c.x4 i.t i.t#i.id) , fe vce(robust)

[Pranshu Tripathi]

Thanks, Clyde.
Your explanation is comprehensive.

[Jeff Wooldridge]

I'm a bit confused about the model you hope to estimate. If id is the cross-sectional identifier and t is the time period identifier, you cannot include i.t##i.id because you will have no separate variation in the variable treat. You can't have i.t and i.t#i.id if you are also using fixed effects, because that's the same as including i.id. So I'd be surprised if you were actually able to estimate the model and get anything useful. I'm guessing that treatment is defined by being in a certain group and there is a post period. Then you can do what Clyde helpfully suggests but you would only have i.t, at most. You can't also have i.t#i.id.

[Pranshu Tripathi]

Dear Jeff,
Actually, i.id here is i.iid (mentioned in first post), which is industry id, not firm id.
Thanks

[Jeff Wooldridge]

That makes sense!