Identification strategy

[Kanika Dua]

Hello everyone,

I need to assess how an intervention that started in 2014 has affected the health of children aged 0-5 years. The challenge is that there's no control group available because the program was implemented nationwide all at once. I can do a pre and post-analysis, but I want to do something more.

I have access to data from three surveys conducted in 2005, 2013, and 2019. Each survey records the health parameters of children aged 0-5 years. For example, the 2005 survey records the health parameters of children born from 2000 to 2005, the 2013 survey records the parameters of children born from 2008 to 2013, and the 2019 survey records the parameters of children born from 2014 to 2019.

Here's my question: Can I use children aged 0-5 years in 2005 and 2013 as a control group since they weren't exposed to the intervention, and consider children aged 0-5 years in 2019 as the treatment group because they were born during periods when the intervention was active? Does this make sense? Or am I missing something here?

Thanks in advance!

[George Ford]

You'd have to qualify the results for the potential of something else happening between 2013 and 2019 other than the treatment. It's not a super compelling result, but you can obviously estimate the coefficient. It is not DID.

[Kanika Dua]

Hello,

Thank you for the very prompt response. Can you please clarify more on "You'd have to qualify the results for the potential of something else happening between 2013 and 2019 other than the treatment" ? Do you mean that I would have to control for other factors that may also affect the dependent variable?

I would be grateful for further clarification.

Thank you once again,

Regards

[Clyde Schechter]

Can you please clarify more on "You'd have to qualify the results for the potential of something else happening between 2013 and 2019 other than the treatment" ? Do you mean that I would have to control for other factors that may also affect the dependent variable?

George Ford seems to be off line now, so I'll jump in here. I'm pretty sure his answer would be the same as mine. Yes, when, as you propose here, you use historical controls, you have to consider the possibility (and in health, the strong likelihood) that something other than the intervention happened between the two time periods that could account for the observed differences.

If you have data on some of those possibilities, then, yes, you should adjust your analysis for variables that changed between the two periods and might have affected the outcome variable(s), assuming your sample size will support such an analysis. But even after adjusting for everything you can, in discussing your findings, you should also acknowledge that other things not included in your analysis might also have confounded the results, and, consequently, you cannot draw firm causal conclusions about the intervention effect.

[Kanika Dua]

Thank you so much George, and Clyde. The clarification is much appreciated.

[George Ford]

What Clyde said.

It might help your case (a bit) if there was no change between 2005 and 2013, which would suggest there is no apparent "trend" in the outcome. That's not a super strong argument, but it is an argument.

[Kanika Dua]

Thank you, George. I had the same hunch!