Difference in Difference analysis

[Lubaba Prottoy]

Hello.

I have 3 period of data, 2012, 2015 and 2018. This is a panel data set. Households in 2012 were asked whether they faced any health shock in the 5 years before the survey. In 2015, they were asked if they faced any health shock since 2012. In 2018 they were asked if they faced any health shock since 2015. The treatment variable in difference in difference analysis would be the health shock. Taking 1 if households faced health shock and 0 otherwise. My outcome variable is consumption of households.

I am fairly new in this and do not possess much knowledge on this type of analysis. Hence, quite a lot of confusions. I would be grateful if someone clears them out.

My question is should I consider 2 years groups for the analysis? Such as 2012 and 2015, 2015 and 2018, 2012 and 2018, in this manner? Is there any way that takes into account all 3 years in difference in difference analysis?

A major source of confusion for me is whether I have to consider those households that were untreated in the baseline as in did not face any health shock? (considering I perform the DD analysis for say 2012 and 2015) In that way the treatment variable would be 1 only if they faced health shock in 2015. Is this the right approach? Dropping households that faced health shock in 2012?

Thank you.

[Lubaba Prottoy]

Can anyone help me with this?

[George Ford]

Interesting problem.

For starters.

(1) dump all treated in 2012. Get the DD estimator for 2015 excluding all the 2018 data. If this was 2017, then that's the model you'd be able to estimate. I'd find this believable (aside from any non-random assignment of the treatment, but health shocks are often quite random, depending on what type of shock you are looking at).

(2) dump all treated through 2015, and get the DD estimator for 2018.

(3) dump all treated in 2012, then estimate a DID coefficient for a 2015 treatment and a 2018 treatment (2 DID coefficients; you may have some treated in both periods, which is where some issues may arise).

This will give you a flavor of what to expect.

Then you can use csdid or jwdid to estimate across all periods (depending on how you specify, it will ignore the past treated in various ways).

Depending on what type of shocks there are in the data, people who engage in risky behaviors may be more at risk, so some matching may be useful, which csdid/jwdid can do, and you can do that for (1) as well.

Ideally, all these will provide similar results. If not, the exercise should help guide you in how to proceed.

[Lubaba Prottoy]

Thank you for your response.
I am sorry if I have failed to grasp some of it. Just for clarification, the numbers (1) (2) (3) indicate different approaches I can take i.e. indicates separate DID specifications?

(1) In that case I am working with 2012 and 2015, dropping all treated in 2012, the post variable is 1 if 2015 and treatment is 1 if treated in 2015. The DD coefficient would be 2015*treated?

(2) In this case I drop all the treated in 2012 and 2015, post is 1 if 2018 and treatment 1 if treated in 2018. Then DID coefficient is 2018*treated?

(3) This I failed to understand. So in the regression model I will have two DID coefficients? 2015*treated and 2018*treated? This can be estimated using csdid?

For (1) and (2) is drdid command suitable?
I was thinking about the matching. Can I use propensity score matching approach in (1) and (2) before running DID. If csdid is used then does it automatically does the matching?

I apologize if some questions are redundant.

[George Ford]

Something like this:

*(1) 
egen treatedin12 = max(cond(year==2012,treat,.)), by(id)
egen treatedin15 = max(cond(year==2015,treat,.)), by(id)


reg y c.post#c.treat post treat  if !treat2012 & year!=2018

*or 

reghdfe c.post#c.treat if !treatedin12 & year!=2018 , absorb(id year)

*(2) 

reg y c.post#c.treat post treat  if !treat2012 & !treatedin15

reghdfe c.post#c.treat if !treatedin12 & !treatedin15 , absorb(id year)

[Lubaba Prottoy]

I did the DID analysis for 2012 and 2015 that is (1), I dumped those households treated in 2012, so that treat is 0 for all 2012 observations. Now when I did the regression the did estimator treated*2015 was omitted because of collinearity. This makes sense because the interaction term and the treatment is always same. So should I not drop the baseline treated observations?

[George Ford]

No. There's something else wrong. It may be in my code.

Check the variables created to make sure they are doing what you want them to.

The treated variable should be 0 or 1 for all periods. Any treated unit is 1 for all years.

Its the treated#post that gets you the DID coefficient, creating 0/1s over time.

The regression should have 2 years (2012, 2015). The mean of treated should be 0 in 2012, and >0 in 2015.
Run this regression as it will signal any issues: tabstat y post treat if !treatedin2012 & year!=2018 , by(year) reg y c.post#c.treat post treat if !treatedin2012 & year!=2018

[Lubaba Prottoy]

HH	Year	treat	post	treat*post
1	2012	0	0	0
1	2015	1	1	1
2	2012	0	0	0
2	2015	0	1	0
3	2012	0	0	0
3	2015	1	1	1
4	2012	0	0	0
4	2015	1	1	1
5	2012	0	0	0
5	2015	0	1	0

Here's a sample table. No household is treated in 2012 i.e. treat is always 0 for 2012, post is always 1 if year is 2015. The collinearity I think occurs between treat and treat*post, even if variation in treatment by households exist.

This is how the dataset looks like. Is this construction wrong?

[George Ford]

what regression did you run?

[George Ford]

treat must be 1 for all periods for any treated unit.

egen treated = max(treat), by(HH)

reg y c.treated#c.post treated post

[Lubaba Prottoy]

The table I provided is part of the dataset after dropping all treated in 2012, then constructed a panel. I then run this regression:

reg y treat post treat*post

treat*post gets omitted. The diff command provides result where Before and After, the difference between treated and control is the same!! I don't know if that is due to some coincidence.

I then did what you said, the treat is same for every treated unit so that the "treated" takes on 1 for a household for both 2012 and 2015 if it is treated in 2015. (I don't understand why do I have to do this? I have declared my dataset as panel)

In this case I obtain results by running
reg y treated post post*treated
But this result is very different from when I use the diff command in this case.

[George Ford]

Defining it as a panel is unhelpful. You can run the same model with pooled data.

As you did it, treat is the DID variable (= 1 when treated and post). That's why you are losing a coefficient.

Drop the treated in 2012.

Define a variable treated15 = 1 for all years if treated in 2015.

dump the 2018 data, run

reg y c.treated15*c.post treated15 post

The treated15 variable gives you the difference in means in the pre-treatment period. The post gives you the change in the mean during the treated period. c.treated#c.post gives you the difference in the change in the means during the post period for the treated units.

Here's the basics:

Y = a + b*treated#post + c*treated + d*post

YC0 = a
YT0 = a + c

YC1 = a + d
YT1 = a + b + c + d

The DID is (YT1 - YT0) - (YC1 - YC0) =
[(a + b + c + d) - (a + c)] - [(a + d) - a]

Do the algebra, you get:
DID = [(a - a) + (c - c) + (b + d)] - [(a - a) + d]
DID = (b + d) - d
DID = b

[Lubaba Prottoy]

Thanks!
so, if my outcome variable is food expenditure, the DID coefficient would indicate the effect of health shock on food expenditure for those who faced the health shock, right?

[George Ford]

Yes