Interpreting csdid results: estat simple, event, and pretrend

[Nate Carroll]

Hello!

I’m hoping to get assistance understanding the differences in some of the results from the very helpful csdid command provides.

In particular, I'm wondering:

1. What is the difference in the “post_avg” result from the “estat event” command and the “ATT” result from the “estat simple” command? Is the difference just that the first (post_avg) is estimated using the sample-size based weights for all the post implementation dynamic periods (i.e. t0, t1, t2…) while the second (simple ATT) is estimated using weights for all groups, for all post periods?
2. It seems like you could assess parallel trends using either the significance from the “pre_avg” result listed after the estat event study command, or by using the test produced by the estat pretrends command. What’s the difference between these two? Is it that the “pre_avg” p-value refers to a test of the weighted average of pre-period effects, and the pretest p-value is a test of the joint significance of all pre-period effects?

If more context, and sample results are helpful, I am studying the impact of a government program for select hospitals. Participation begins at different times for different hospitals, so I’m conducting a difference in differences analysis with staggered implementation.

To estimate effects I use the csdid command with the dripw option.
The full command, including covariates and specifying appropriate subgroups of the data, is

csdid births bdtot i.own i.sys i.coth cmi alldc hhi i.medicaidexp psr unempr uninsr income ppov alldc mcddc mcrdc if (entitytype==1 | entitytype==6) & (partgrp==4 | partgrp==6) & cah==0 & rural == 0, time(year) gvar(yrstart) method(dripw) cluster(mcrnumbyte)

I aggregate results using

Estat simple
Estat event

And get the following results:

. estat simple
Average Treatment Effect on Treated
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
ATT | -117.9978 374.4133 -0.32 0.753 -851.8344 615.8387
------------------------------------------------------------------------------

. estat event
ATT by Periods Before and After treatment
Event Study:Dynamic effects
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
Pre_avg | 119.5955 124.4713 0.96 0.337 -124.3638 363.5548
Post_avg | -76.7968 384.9611 -0.20 0.842 -831.3067 677.7131
Tm5 | -190.9287 105.8804 -1.80 0.071 -398.4505 16.5931
Tm4 | 74.60136 216.3211 0.34 0.730 -349.3801 498.5829
Tm3 | -316.8462 232.5711 -1.36 0.173 -772.6772 138.9849
Tm2 | 1229.28 479.3727 2.56 0.010 289.7267 2168.833
Tm1 | -198.1292 385.5943 -0.51 0.607 -953.8801 557.6218
Tp0 | 1098.908 874.5222 1.26 0.209 -615.1237 2812.94
Tp1 | -388.361 268.1099 -1.45 0.147 -913.8467 137.1247
Tp2 | 42.03581 711.2503 0.06 0.953 -1351.989 1436.061
Tp3 | -193.7215 488.3974 -0.40 0.692 -1150.963 763.5199
Tp4 | 47.31053 447.6242 0.11 0.916 -830.0167 924.6378
Tp5 | 9.043332 675.4317 0.01 0.989 -1314.778 1332.865
Tp6 | 426.5687 543.0358 0.79 0.432 -637.7619 1490.899
Tp7 | -453.0728 549.3245 -0.82 0.409 -1529.729 623.5834
Tp8 | -670.0298 806.9976 -0.83 0.406 -2251.716 911.6564
Tp9 | -2773.461 1113.352 -2.49 0.013 -4955.591 -591.3316
Tp10 | 30.36106 858.0278 0.04 0.972 -1651.343 1712.065
Tp11 | 1902.857 1119.142 1.70 0.089 -290.621 4096.335
------------------------------------------------------------------------------

I test for parallel trends using the command estat pretrend, and get the following results

estat pretrend
Pretrend Test. H0 All Pre-treatment are equal to 0
chi2(15) = 24.7763
p-value = 0.0530

Any insight into the two questions above would be very helpful. Thanks!

[Haykal Hakeem]

I'm sorry, have you got any answer to it? I'm trying to figure out the same thing

[eric_a_booth]

estat simple (ATT) vs. post_avg: estat simple averages every post-treatment group-time effect, weighted strictly by sample size. It's your overall bottom-line effect.
Details: post_avg averages your event-study periods (Tp0, Tp1, etc.). Because your rollout is staggered, early adopters heavily skew this number since they are the only ones who actually reach late periods like Tp11. So, estat simple is ATT, it takes every valid post-treatment ATT(g,t) cell and averages them together, weighting each cell strictly by its sample size. This is your single-number summary of the policy's effect across the entire post-treatment sample. estat event (post_avg) first calculates an ATT for each relative event period (Tp0, Tp1, etc) by averaging the groups that experienced that specific length of exposure. The post_avg result is the average of those post-treatment event-time coefficients.

Yes, to your questions: pre_avg is the average of your pre-period effects; estat pretrend is a joint test asking: "Are ALL pre-periods simultaneously zero?"

Be sure to use estat pretrend to check parallel trends: your own results show exactly why. You have a large spike in period Tm2. that is, estat event table, period Tm2 shows significant coefficient (1229.28, p=0.010). This is a violation of parallel trends. Because the other pre-periods (Tm5, Tm3, Tm1) are mostly negative, they cancel out Tm2 when averaged. This results in a pre_avg p-value of 0.337, which makes it look like there is no issue.pre_avg completely hides the parallel trends violation (p=0.33), but estat pretrend catches the problem (p=0.05).