Interactions in Difference in Differences models

[Tom Ford]

Dear All,

I am estimating a DiD model, and I would like to examine if treatment has different effects across genders. Do you know how I can do it?

My baseline regression is as such:

************
*** TWFE ***
************
reghdfe Y T, absorb(id time) cluster(id)

Where Y is my outcome of interest and T is my post treatment dummy.
How do I examine the difference in effect across subgroups? Is is as simple as adding the interaction term and the control as such?

************
*** TWFE ***
************
reghdfe Y i.T i.T#i.G i.G, absorb(id time) cluster(id)

where G is a dummy for gender? Or is this the wrong approach?


Also consider that I have a staggered design, hence I am also estimating an event plot as such:

reghdfe Y L_T* F_T*, absorb(id time) cluster(id)

How to I include the interaction in this model?

I am aware about the new DiD literature, and the problem of using TWFE estimation, but I do not think the new estimators allow for interactions. is this correct?


Thanks in advance for your suggestions

Best

[Jeff Wooldridge]

Tom: In fact, the newer regression-based methods handle interactions even more flexibly than you've attempted. I think the best way is to use jwdid, which shows the so-called moderating effects. To use this command, you need to define a treatment cohort for each unit. The cohort is the first period of treatment. Never treated units are assigned zero. So, assuming you have a variable called "cohort" defined as the first period of treatment, the following will work:

jwdid Y G, ivar(id) tvar(time) gvar(cohort)

This implements the extended TWFE estimator -- actually, the usual TWFE estimator applied to an extended model -- in Wooldridge (2021). This can be used for common timing or staggered interventions. In the common timing case, the cohort variable is either zero or the first period of the intervention.

[Bruce Weaver]

Note that -jwdid- is a user-written package.

ssc describe jwdid

[Tom Ford]

Dear Prof Wooldridge,
thank you very much for your reply and your helpfulness, this is indeed extremely helpful.

The one thing I am unsure off is whether I am aggregating and interpreting the results correctly. Take the following example

frause mpdta.dta, clear

* I create my dummy for interaction
gen gender = runiform() > 0.5

* run a simple model with a simple interaction:
jwdid lemp gender, ivar(countyreal) tvar(year) gvar(first_treat) group

How do I test whether the effect of the treatment is different across groups? Using simple aggregation and ores() I can estimate the effect for different groups but is there a built in way of examining if these are statistically different from one another? or should I manually calculate the t-statistic?

. estat simple, orestriction(gender==0)
------------------------------------------------------------------------------
             |            Delta-method
             | Coefficient  std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
      simple |  -.0449632   .0130241    -3.45   0.001      -.07049   -.0194364
------------------------------------------------------------------------------

. estat simple, orestriction(gender==1)
------------------------------------------------------------------------------
             |            Delta-method
             | Coefficient  std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
      simple |  -.0504378   .0135822    -3.71   0.000    -.0770584   -.0238172
------------------------------------------------------------------------------

* Compute the t-statistic for difference in coefficients
scalar t_stat = (b0 - b1) / sqrt(se0^2 + se1^2)

Another question I have is that I have noticed that Stata gives this warning:

WARNING: Singleton observations not dropped; statistical significance is biased (link)

is there a way to adress this issue?

thanks a lot in advance for your support

[Jeff Wooldridge]

Hi Tom. I don't know if there is a way to use "estat simple" and have it show you the same raw output -- and, in particular, the weighted moderating effect -- that the original command gives. You could replicate jwdid "by hand," but that's tedious. Your solution for computing the t_stat is valid because when gender is interacted with everything, it's like doing the estimation on the two samples separately. The calculated standard errors already assume independence across units, so there is no covariance between the estimates to worry about as they are based on different subsamples.

Do you have treated cohorts that are very small? If not, you can ignore that error message. It always comes up even when you have no cohorts with just a single treated observation. In the data set you're using, the smallest cohort has 100 treated units. You have to be aware of this in general, though.

[FernandoRios]

Actually that is easy now.
you could simply do
estate simple, over(gender)
and then test the difference between coefficients

[Bruce Weaver]

Note that -jwdid- is a user-written package.

ssc describe jwdid

Ditto for -frause- in #4, in case anyone is wondering.

ssc describe frause

[Tom Ford]

Dear Jeff Wooldridge thank you very much for your support in this matter. It is indeed extremely helpful to know that I can use a t-test to evaluate differences and that I should not worry too much about singletons as my treated cohorts are relatively large with hundreds of observations in each one of them, except for the singletons obviously.

Dear FernandoRios, thank you very much for your suggestions on this matter and more broadly to have deevloped your command that is also very helpful. However I am not entirely sure I am understanding this correctly. I have attempted to use estate simple, over(gender), but it appears to me that it simply reports the overall effect when gender is =1 rather than the interaction effect of gender and treatment.

take the following example>

frause mpdta.dta, clear

set seed 123

* I create my dummy for interaction
gen gender = runiform() > 0.5

* run a simple model with a simple interaction:
qui:jwdid lemp gender, ivar(countyreal) tvar(year) gvar(first_treat) group

. estat simple, over(gender)
------------------------------------------------------------------------------
             |            Delta-method
             | Coefficient  std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
      simple |  -.1241402   .1375639    -0.90   0.367    -.3937604      .14548
------------------------------------------------------------------------------

. estat simple, orestriction(gender==1)
------------------------------------------------------------------------------
             |            Delta-method
             | Coefficient  std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
      simple |  -.1241402   .1375639    -0.90   0.367    -.3937604      .14548
------------------------------------------------------------------------------

`estat simple, over(gender)' and `estat simple, orestriction(gender==1)' provide identical results. How can I test the difference among these coefficients. Ideally I would like to have a coefficient that indicates the overall difference in the effect of treatment across genders and see whether this is significant. Is this possible?


thanks again for your support on this matter

[Andrew Musau]

Deleted. Posted in wrong thread.

[FernandoRios]

I suspect that may be because of Gender Code. Can you code it with Gender = 1 and 2 and try again?

[Tom Ford]

Dear FernandoRios, yes you are correct, there was an issue in how the command dealt with a Gender. If I recode the gender variable, here is what I get:

. set seed 123

. * I create my dummy for interaction
. gen gender = runiform() > 0.5

. replace gender=2 if gender==0
(1,236 real changes made)

. tab gender

     gender |      Freq.     Percent        Cum.
------------+-----------------------------------
          1 |      1,264       50.56       50.56
          2 |      1,236       49.44      100.00
------------+-----------------------------------
      Total |      2,500      100.00

. * run a simple model with a simple interaction:
. qui: jwdid lemp i.gender, ivar(countyreal) tvar(year) gvar(first_treat) group
WARNING: Singleton observations not dropped; statistical significance is biased (link)

. estat simple, over(gender)
------------------------------------------------------------------------------
             |            Delta-method
             | Coefficient  std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
     simple1 |  -.1241402   .1375639    -0.90   0.367    -.3937604      .14548
     simple2 |   .0369503   .1443322     0.26   0.798    -.2459356    .3198361
------------------------------------------------------------------------------

So, to test whether these two coefficient are different I am using lincom, but this provides the wrong text. Given the levels of my simple aggregations over gender, the coefficient of my linear combination cannot be 0.06.

How would you do the test correctly ?

lincom _b[1.gender] - _b[2.gender]

 ( 1)  1b.gender - 2.gender = 0

------------------------------------------------------------------------------
        lemp | Coefficient  Std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
         (1) |   .0631898   .1447975     0.44   0.663    -.2212981    .3476777
------------------------------------------------------------------------------

thank you very much in advance

[FernandoRios]

You forgot "post"
estat simple, over(gender) post
otherwise the "active" estimations are from JWDID

[Tom Ford]

Thanks! this is really helpful!