Using a single database for a Difference-in-Difference analysis with three time periods

[Stephen Butler]

Hello all
I'm assessing the impact of an intervention over the course of two years (from 2016-17, and from 2017-18).
I'm using a difference-in-difference analysis, comparing the treatment group to a control group over three years: pre-treatment (2015-16) and post-treatment (2016-17 and 2017-18).

I'd like to calculate the DiD in coefficient for both the change over 2016-17 and then 2017-18. I can do this simply by separating my data into two databases, with 2 time points each (one to compare 2015-16 to 2016-17, and a separate database to compare 2016-17 to 2017-18). In each case, I have a dummy variable which equals 0 in the earlier year, and 1 in the later year. I also have a dummy for whether the variable is in the treated or control group. I then regress the outcome variable over Time##Treated.

I would like to do both comparisons using a single database, as this would be more succinct and less clumsy. I'd also like to show the cumulative effect of the intervention over the two years. Is it possible?
Many thanks.

[Clyde Schechter]

I would like to do both comparisons using a single database, as this would be more succinct and less clumsy.

To which I would add, a more statistically solid analysis.

So the first thing is to forget about your separate indicator ("dummy") variables for 2016-17 and 2017-18. Instead what you need is a three level variable that is 0 for observations from 2015-2016, 1 for observations from 2016-2017, and 2 for observations from 2017-2018. Let's call that variable era. Then your regression model becomes:

regression_command outcome_variable i.era##i.Treated perhaps_other_variables

Then you can get things like the predicted outcome in each group in each era:

margins era#Treated

Or you can get the difference between expected outcomes in each group in each era:

margins Treated, dydx(era)

[Stephen Butler]

Thanks Clyde.
I can calculate the predicted outcomes for each group, in each year. However, 'margins' doesn't seem to calculate the interaction.

This is my regression output:

perf | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
Year |
2017 | -1.063955 .6237676 -1.71 0.088 -2.286557 .1586483
2018 | -1.243376 .6294985 -1.98 0.048 -2.477212 -.0095408
|
1.treat | 22.55029 .6320399 35.68 0.000 21.31147 23.78911
|
Year#treat |
2017 1 | -4.784684 .647142 -7.39 0.000 -6.053101 -3.516267
2018 1 | -12.53964 .6528977 -19.21 0.000 -13.81934 -11.25994
|



But when I do 'margins Treated, dydx(Year), I get:


Expression : Linear prediction, predict()
dy/dx w.r.t. : 2017.Year 2018.Year

------------------------------------------------------------------------------
| Delta-method
| dy/dx Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
2016.Year | (base outcome)
-------------+----------------------------------------------------------------
2017.Year |
treat |
0 | . (not estimable)
1 | . (not estimable)
-------------+----------------------------------------------------------------
2018.Year |
treat |
0 | . (not estimable)
1 | . (not estimable)
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.


I'm not sure why this would be?

[Stephen Butler]

Hi Clyde, Apologies I finally got dydx() to work.
However, it only shows the changes for each year compared to the baseline. I was also looking to compare 2018 to 2017 (separately from 2017 compared to 2016).
Do you think this is possible? I imagine I need to do a slightly different regression analysis.
Best wishes
Stephen

. margins treat, dydx(Year)

Average marginal effects                        Number of obs     =     36,694
Model VCE    : OLS

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 2017.Year 2018.Year

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
2016.Year    |  (base outcome)
-------------+----------------------------------------------------------------
2017.Year    |
       treat |
          0  |  -1.064591   .6974591    -1.53   0.127    -2.431631    .3024497
          1  |  -5.842027   .1945383   -30.03   0.000    -6.223328   -5.460726
-------------+----------------------------------------------------------------
2018.Year    |
       treat |
          0  |  -1.246701    .703867    -1.77   0.077    -2.626301     .132899
          1  |  -13.78112   .1970704   -69.93   0.000    -14.16739   -13.39486
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

[Clyde Schechter]

To compare 2017 with 2018, try -margins treat, dydx(Year) pwcompare-

[Stephen Butler]

Yes that works perfectly, thanks for the suggestion Clyde.