Diff-in-Diff analysis

[Sophie Polo]

Dear all,

I ran a differences-in-differences regression with control variables to estimate the purchase frequency of customer measured in days between two purchases. I have a treatment group and a control group and a before and after period (policy introduction).
My regression equation:


Flatmember (Treatment) : Treatment =1 or not =0
Timperiod : Time =1 after treatment, =0 before treatment
did: Interaction Term

Can you please help me on interpreting the results? According to the attached output, all coefficients of interest (Flatmember, Timeperiod did) are highly significant.

Thank you for your help!

Regards Carina

[Sam Sellers]

Hi Sophie!

I would recommend looking at some online resources for DID analysis first. For instance, Columbia has a nice page with a table showing the interpretations for each of the key coefficients (Treatment, Time, and Treatment*Time).
https://www.mailman.columbia.edu/res...nce-estimation

Regards,
Sam

[River Huang]

Dear Sophie, I don't think that your Stata code is correct for doing DID analysis. I'd suggest that you "search diff" in Stata command window. Please have a look at the help file, and the accompanying paper (https://www.stata-journal.com/articl...article=st0424).

[Tarjei W. Havneraas]

Hi Sophie

I would encourage you to use Stata's indicators to specify your DID analysis. It looks as if you've generated a -did- variable and included it in your regression. You could alternatively specify your model as -xtreg Frequency i.Flatmember##i.TimePeriod Discounts Returns CAge, r-. I assume the use of random effects to account for clustering is motivated.

The estimand in DID is usually the average treatment effect on the treated (ATET), which is the effect of the treatment on the units that received the treatment. Morgan & Winship (2015) has more to say on ATET - and there's a ton of other good online resources for DID.

I'm not sure about the interpretation of your variables as I don't know your dataset, but the DID-estimate indicate that the treatment effect is 128.84, so this is the increase among units that received treatment. Normally, the binary prepost-variable and the group-variable are not so interesting in themself, neither are control variables (see e.g. Table 2 fallacy). Again, I would recommend you to rerun the analysis using the indicator-notation.

Last time I checked, the -diff- command had some issues in panel data settings. Clyde Schechter (on this forum) has some good input on -diff- and he also has several posts on interpretation of DID (can't recall the relevant links).

Lastly, I would not necessarily stress "significance", but rather the substansive interpretation of the effect size and the confidence intervals, but this is a somewhat sidenote here - see e.g. ASA statement on p-values.