Difference in differences with one regression totally insignificant

[John Snow]

Hello everybody,

as the name of the thread suggests, Im currently trying to do a diff-in-diff analysis for my treatment group (some EU countries). The event is the European debt crisis starting in 2009 and therefore the periods are defined as 2001-2008 and 2009-2012. However, when I run the fixed effect regression for 2009-2012, all coefficients get insignificant. So my question now is if I still can do a diff-in-diff analysis and what could be the causes of this insignificance? do I have not enough observations for the 2009-2012 period?

Here are the regression results:

Before the crisis

Thank you for your help !

[Clyde Schechter]

Yes, you can. I don't understand why you even raise the question. The statistical significance of variables in either time period has nothing to do with it at all. A difference in differences analysis looks at the difference between the two groups (whatever they may be in your situation, I guess groups of countries that did and didn't experience the European crisis) in the difference in outcome trends from before and after the event. There is no requirement that any of those trends be significant or not in either time period. Either way, what is sought is the change from before to after in each group, and, ultimately, the difference between those changes. The era-specific regressions you show above shed no light on that.

You do raise the specific question of whether you have enough observations for the 2009-2012 period, but you don't tell us how many you have! Now, it isn't surprising that there is less data available in a 4 year period than in an 8 year period. And looking at the coefficients in the two regressions we see that most of them are as large or larger in magnitude in the post-crisis regression. So it is likely that a smaller sample size explains why these as large or larger coefficients are associated with less impressive p-values. But there may be more to it than that. The R-square is lower in the later period. If the associations were just as strong in both eras, I would not expect the R-square to decrease, as it is not sample-size sensitive (or, rather, to the extent it is, smaller sample sizes can result in larger R-squares.)

But whatever is going on here, I don't see it as an obstacle to running a difference-in-differences analysis unless these results suggest to you that your data is actually erroneous.