difference between DD and DDD estimates

[jonno turner]

Hi guys,

I'm running a d-i-d regression on how a public program affected the healthcare of a group of people in a village. The healthcare program was general in nature except de facto it really only affected a specific group of people. I have the following issues and if anyone could help me that would be great. Research design basically uses two bordering villages, one who received the treatment the other did not. In the treatment village there is the `de facto' group, and the 'non-de facto group'. Call these group DF and group NDF respectively

The tricky situation is the how the de facto treatment group fits in here.

1. I probably want to run a DDD regression only using group DF as the treatment group
2. I have been advised to run two DD regressions before I go straight to the DDD. The first DD regression I run is: everyone in the control village vs everyone in the treatment village (DF+NDF).
3. The second DD regression I run is: everyone in the de-facto group in the treatment village vs. everyone in the non de-facto group in the treatment village. (FD VS NDF).
4. Then I run the DDD which only uses everyone in the de-facto treatment group as the treatment group, and all those in the control village + non de-facto group in the `treatment village' are my controls.

I'm getting some strange results.

1. The first DD regressions tells me there was no significant change (point estimate of about 0)
2. The second DD regression tells me there was a significant change (point estimate of about 2)
3. The DDD regression tells me there was a significant change. (point estimate of about 4)

So there was no overall treatment effect if I just compare everyone in the treatment vs everyone in the control village.
There was a treatment effect if I just compare everyone in groups DF and NDF
There was a treatment effect if I use group DF as the treatment, and everyone else as the control.

I'm having trouble interpreting these results. Is it that in the first regression, including group NDF as treated is dampening the true treatment effect?
The second regression reveals that there was a treatment effect between group NDF and group DF
The DDD regression reveals there was an overall treatment effect using just the DF as the treatment group.

The second point is, what does it mean for the DDD to be almost double in size of the second DD regression? I don't think this is possible, given the no significant result of the first DD regression

Any comments will be appreciated.

thank you for you time

[jonno turner]

I should add that the de-facto group is a specific ethnicity, and the control village also has people of that ethnicity. So the DDD model will have the indicator (ethnicity*village*treatment period)

[Phil Bromiley]

You didn't get a quick answer. You'll increase your chances of a useful answer if you follow the FAQ on asking questions - provide Stata code in code delimiters, Stata output, and sample data using dataex. Also try to cut your code to what you need to demonstrate your problem.

Instead of a Stata problem, you've given us a very complicated design description. Folks on the list will often help with them, but they need to be clearer. Since you don't provide code, no one can be sure what you really ran. Every time you change the groups you use, you should expect a change in results. I'm not sure why you have what you call a DDD estimator.