[Rattiya Lippe]

Hi Rattiya
That is not necessary the case
You need to do a careful analysis of the "event" dynamics, and use that as additional evidence to say whether or not parallel trends hold
Also, you may need to control for other factors, or consider different function specifications (logs? )
HTH
F

Hi FernandoRios , thank you. I ran the "estat event" postestimation, and the result is as follow:

	Coefficient	Std. err.	z	P>z	[95% conf.	interval]
Pre_avg	0.000	0.000	-2.480	0.013	0.000	0.000
Post_avg	0.000	0.000	0.140	0.885	-0.001	0.001
Tm25	0.000	0.000	1.510	0.132	0.000	0.000
Tm24	0.000	0.000	-2.530	0.011	0.000	0.000
Tm23	0.000	0.000	1.460	0.143	0.000	0.000
Tm22	0.000	0.000	-1.030	0.301	0.000	0.000
Tm21	0.000	0.000	1.820	0.069	0.000	0.000
Tm20	0.000	0.000	-1.420	0.157	0.000	0.000
Tm19	0.000	0.000	-0.510	0.611	0.000	0.000
Tm18	0.000	0.000	-0.860	0.391	0.000	0.000
Tm17	0.000	0.000	1.070	0.283	0.000	0.000
Tm16	0.000	0.000	0.100	0.924	0.000	0.000
Tm15	0.000	0.000	-0.100	0.917	0.000	0.000
Tm14	0.000	0.000	1.070	0.283	0.000	0.000
Tm13	0.000	0.000	-1.990	0.046	0.000	0.000
Tm12	0.000	0.000	-2.370	0.018	0.000	0.000
Tm11	0.000	0.000	3.830	0.000	0.000	0.000
Tm10	0.000	0.000	0.590	0.558	0.000	0.000
Tm9	0.000	0.000	-2.820	0.005	0.000	0.000
Tm8	0.000	0.000	1.870	0.061	0.000	0.000
Tm7	0.000	0.000	-0.330	0.740	0.000	0.000
Tm6	0.000	0.000	-1.130	0.260	0.000	0.000
Tm5	0.000	0.000	0.060	0.949	0.000	0.000
Tm4	0.000	0.000	-1.470	0.143	0.000	0.000
Tm3	0.000	0.000	-1.060	0.288	0.000	0.000
Tm2	0.000	0.000	0.850	0.397	0.000	0.000
Tm1	0.000	0.000	-2.130	0.033	-0.001	0.000
Tp0	0.000	0.000	0.990	0.322	0.000	0.001
Tp1	0.000	0.000	1.740	0.082	0.000	0.000
Tp2	0.000	0.000	0.110	0.909	-0.001	0.001
Tp3	0.000	0.000	-1.030	0.303	0.000	0.000
Tp4	0.000	0.000	-0.690	0.489	-0.001	0.000
Tp5	0.000	0.000	0.090	0.930	-0.001	0.001
Tp6	0.001	0.002	0.550	0.582	-0.002	0.004
Tp7	0.000	0.001	0.500	0.618	-0.001	0.002
Tp8	0.000	0.001	0.020	0.984	-0.001	0.001
Tp9	-0.001	0.000	-2.510	0.012	-0.001	0.000
Tp10	0.000	0.001	-0.310	0.760	-0.002	0.001

Do I understand correctly that: some coefficients of the pre-treatment years (Tm) are significant, implying that there are significant differences between treatment and control groups in the pre-treatment period? And this also confrims that why the pararell trend assumption is not held in my case?..

Kind regards, Rattiya

[Ayu Fitriani]

Thanks FernandoRios .
One more question, so I have run the code and for this following result.


Number of obs = 10,416
Outcome model : least squares
Treatment model: inverse probability
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
g2000 |
t_2000_2007 | 0 (omitted)
t_2000_2014 | 0 (omitted)
-------------+----------------------------------------------------------------
g2007 |
t_2000_2007 | 1.149023 .0673463 17.06 0.000 1.017026 1.281019
t_2000_2014 | .5029631 .0807806 6.23 0.000 .3446359 .6612902
-------------+----------------------------------------------------------------
g2014 |
t_2000_2007 | .0408478 .059203 0.69 0.490 -.0751879 .1568834
t_2007_2014 | .7401725 .0449899 16.45 0.000 .6519939 .8283512
------------------------------------------------------------------------------
Control: Never Treated


and then I tried to graph this by "events",

csdid_stats event, wboot rseed(1)
---------------------------------------------------------------------
| Coefficient Std. err. t [95% conf. interval]
------------+--------------------------------------------------------
Pre_avg | .0408478 .0588727 0.69 -.1047298 .1864253
Post_avg | .7007282 .0533918 13.12 .5687035 .8327528
Tm7 | .0408478 .0588727 0.69 -.1047298 .1864253
Tp0 | .8984933 .0425183 21.13 .7933563 1.00363
Tp7 | .5029631 .0783518 6.42 .3092186 .6967075
---------------------------------------------------------------------

Can you tell me what is the mean of pre_avg, post_avg, Tm7, Tp0, and Tp7?

And by "Group"...

csdid_stats group, wboot rseed(1)

---------------------------------------------------------------------
| Coefficient Std. err. t [95% conf. interval]
------------+--------------------------------------------------------
GAverage | .7734051 .040387 19.15 .6827671 .8640432
G2007 | .825993 .0685029 12.06 .6722563 .9797296
G2014 | .7401725 .0459832 16.10 .6369755 .8433696
---------------------------------------------------------------------

What is the mean of GAverage here? Because I think it should be G2000, but I don't have my Group 2000 estimation in the first table above.

Thank you in advance!

Best,

Ayu

[FernandoRios]

Kind regards, Rattiya

Hi there
The results you show are very odd. So even tho you are making a reasonable interpretation of the results, most coefficients seem t obe zero. So there is nothing to interpret!
Perhaps you copy pasted the wrong results?
F

[FernandoRios]

Thanks FernandoRios .
One more question, so I have run the code and for this following result.


Number of obs = 10,416
Outcome model : least squares
Treatment model: inverse probability
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
g2000 |
t_2000_2007 | 0 (omitted)
t_2000_2014 | 0 (omitted)
-------------+----------------------------------------------------------------
g2007 |
t_2000_2007 | 1.149023 .0673463 17.06 0.000 1.017026 1.281019
t_2000_2014 | .5029631 .0807806 6.23 0.000 .3446359 .6612902
-------------+----------------------------------------------------------------
g2014 |
t_2000_2007 | .0408478 .059203 0.69 0.490 -.0751879 .1568834
t_2007_2014 | .7401725 .0449899 16.45 0.000 .6519939 .8283512
------------------------------------------------------------------------------
Control: Never Treated


and then I tried to graph this by "events",

csdid_stats event, wboot rseed(1)
---------------------------------------------------------------------
| Coefficient Std. err. t [95% conf. interval]
------------+--------------------------------------------------------
Pre_avg | .0408478 .0588727 0.69 -.1047298 .1864253
Post_avg | .7007282 .0533918 13.12 .5687035 .8327528
Tm7 | .0408478 .0588727 0.69 -.1047298 .1864253
Tp0 | .8984933 .0425183 21.13 .7933563 1.00363
Tp7 | .5029631 .0783518 6.42 .3092186 .6967075
---------------------------------------------------------------------

Can you tell me what is the mean of pre_avg, post_avg, Tm7, Tp0, and Tp7?

And by "Group"...

csdid_stats group, wboot rseed(1)

---------------------------------------------------------------------
| Coefficient Std. err. t [95% conf. interval]
------------+--------------------------------------------------------
GAverage | .7734051 .040387 19.15 .6827671 .8640432
G2007 | .825993 .0685029 12.06 .6722563 .9797296
G2014 | .7401725 .0459832 16.10 .6369755 .8433696
---------------------------------------------------------------------

What is the mean of GAverage here? Because I think it should be G2000, but I don't have my Group 2000 estimation in the first table above.

Thank you in advance!

Best,

Ayu

Hi Ayu]
1. tmX tpX stand for T-X or T+X. Where X is the relative period to the treatment. In your case, with data spanning 7 years apart, it showing Tp7
2. Avg is as the name suggest, the Average of all Pre or Post treatment ATTs
3 GroupAvg are the group Averages. See Callaway and Sant'Anna paper to see the exact definitions. There is no G2000 , because you never observe data Before 2000 thus no ATT can be estimated for that cohort
F

[Rattiya Lippe]

Hi there
The results you show are very odd. So even tho you are making a reasonable interpretation of the results, most coefficients seem t obe zero. So there is nothing to interpret!
Perhaps you copy pasted the wrong results?
F

Hi FernandoRios , yes.. the coefficients are really small. I increased the decimal as follows:

	Coefficient	Std. err.	z	P>z	[95% conf.	interval]
Pre_avg	-0.000110	0.000068	-1.6200	0.1050	-0.0002	0.0000
Post_avg	-0.002239	0.001708	-1.3100	0.1900	-0.0056	0.0011
Tm25	0.000021	0.000020	1.0500	0.2920	0.0000	0.0001
Tm24	-0.000008	0.000015	-0.5500	0.5800	0.0000	0.0000
Tm23	-0.000032	0.000044	-0.7100	0.4760	-0.0001	0.0001
Tm22	0.000042	0.000041	1.0100	0.3130	0.0000	0.0001
Tm21	-0.000066	0.000067	-0.9900	0.3230	-0.0002	0.0001
Tm20	0.000033	0.000068	0.4800	0.6290	-0.0001	0.0002
Tm19	-0.000036	0.000050	-0.7200	0.4710	-0.0001	0.0001
Tm18	0.000005	0.000064	0.0700	0.9430	-0.0001	0.0001
Tm17	-0.000122	0.000081	-1.5000	0.1340	-0.0003	0.0000
Tm16	0.000059	0.000100	0.5900	0.5570	-0.0001	0.0003
Tm15	0.000068	0.000079	0.8600	0.3900	-0.0001	0.0002
Tm14	-0.000085	0.000082	-1.0400	0.2970	-0.0002	0.0001
Tm13	-0.000151	0.000087	-1.7300	0.0840	-0.0003	0.0000
Tm12	-0.000165	0.000166	-0.9900	0.3200	-0.0005	0.0002
Tm11	0.000157	0.000136	1.1600	0.2480	-0.0001	0.0004
Tm10	-0.000078	0.000137	-0.5700	0.5690	-0.0003	0.0002
Tm9	-0.000706	0.000602	-1.1700	0.2410	-0.0019	0.0005
Tm8	0.000448	0.000427	1.0500	0.2940	-0.0004	0.0013
Tm7	-0.000318	0.000221	-1.4400	0.1510	-0.0008	0.0001
Tm6	0.000205	0.000146	1.4000	0.1600	-0.0001	0.0005
Tm5	0.000097	0.000166	0.5900	0.5580	-0.0002	0.0004
Tm4	-0.001126	0.000643	-1.7500	0.0800	-0.0024	0.0001
Tm3	-0.000012	0.000466	-0.0300	0.9790	-0.0009	0.0009
Tm2	0.000074	0.000405	0.1800	0.8560	-0.0007	0.0009
Tm1	-0.001040	0.000588	-1.7700	0.0770	-0.0022	0.0001
Tp0	-0.000464	0.001162	-0.4000	0.6900	-0.0027	0.0018
Tp1	-0.001430	0.001402	-1.0200	0.3080	-0.0042	0.0013
Tp2	-0.000400	0.000592	-0.6800	0.5000	-0.0016	0.0008
Tp3	-0.000861	0.000807	-1.0700	0.2860	-0.0024	0.0007
Tp4	-0.001600	0.001249	-1.2800	0.2000	-0.0040	0.0008
Tp5	-0.001566	0.001517	-1.0300	0.3020	-0.0045	0.0014
Tp6	-0.003787	0.002764	-1.3700	0.1710	-0.0092	0.0016
Tp7	-0.004620	0.003478	-1.3300	0.1840	-0.0114	0.0022
Tp8	-0.002031	0.002099	-0.9700	0.3330	-0.0061	0.0021
Tp9	-0.003506	0.002130	-1.6500	0.1000	-0.0077	0.0007
Tp10	-0.004369	0.002822	-1.5500	0.1220	-0.0099	0.0012

Best, Rattiya

[FernandoRios]

In that case, My original comment Stands
But what is more important. It seems your treatment has no impact on the outcome. Of if any, is too small.

[Rattiya Lippe]

In that case, My original comment Stands
But what is more important. It seems your treatment has no impact on the outcome. Of if any, is too small.

Thank you very much.. I totally agree..
Rattiya

[Bruno Paisani]

Hello FernandoRios!

Please, I have a question regarding the propensity score matching and the use of CSDID command.

1) Is the matching being done "behind the scenes" when I simply run CSDID with X covariates OR do I have to do it beforehand and then run CSDID taking covariates into consideration?

2) In the last case, how can I do that if I have panel data?


I am sorry if these doubts are too basic.

Best Regards.
Bruno.

[FernandoRios]

Hi Bruno
No need. if you use Method(dripw) or method(drimp), csdid automatically runs all the relevant logits or IPTs to estimate the Pscores for each ATTGT.
So there is no need for you to do something else.
F

[Bruno Paisani]

Hi Bruno
No need. if you use Method(dripw) or method(drimp), csdid automatically runs all the relevant logits or IPTs to estimate the Pscores for each ATTGT.
So there is no need for you to do something else.
F

Great!
Thanks a lot.
Bruno.

[Rattiya Lippe]

Dear FernandoRios , Please I have some questions on understanding the results after running the csdid. My apologize in advance if these questions are too basic.

1. The result of pretend test is shown below. Do I understand correctly that the result indicates that the pararell trend is held?

Pretrend Test. H0 All Pre-treatment are equal to 0
chi2(116) = 122.1841
p-value = 0.3290

2. In my anaylysis, I have 5 cohorts indicates the first year of being treated. The results is as follow: How do I interpret "GAverage coefficient"?

Average Treatment Effect on Treated
Coefficient Std. err. z P>z [95% conf. interval]

ATT -.0020791 .001009 -2.06 0.039 -.0040567 -.0001015



Coefficient Std. err. z P>z [95% conf. interval]

GAverage -.0013213 .0008718 -1.52 0.130 -.0030301 .0003874
G2005 -.0031371 .0011438 -2.74 0.006 -.0053789 -.0008952
G2008 -.0039495 .0015517 -2.55 0.011 -.0069907 -.0009082
G2011 -.0029748 .0016319 -1.82 0.068 -.0061734 .0002237
G2012 -.0043824 .0016268 -2.69 0.007 -.0075708 -.001194
G2015 .0037665 .0027934 1.35 0.178 -.0017085 .0092415
G2017 -.000036 .001475 -0.02 0.981 -.002927 .002855


3. I also like to report the number of observations by cohorts. However, the number of control groups are not the same in one cohort. For example, the no. of control group drop from 596 to 586 , and then later to 546. How can we explain this situation?

e(gtt)[180,7]
r31 2008 1991 1992 0 642 626 16
r32 2008 1992 1993 0 642 626 16
r33 2008 1993 1994 0 642 626 16
r34 2008 1994 1995 0 642 626 16
r35 2008 1995 1996 0 642 626 16
r36 2008 1996 1997 0 642 626 16
r37 2008 1997 1998 0 642 626 16
r38 2008 1998 1999 0 642 626 16
r39 2008 1999 2000 0 642 626 16
r40 2008 2000 2001 0 642 626 16
r41 2008 2001 2002 0 642 626 16
r42 2008 2002 2003 0 642 626 16
r43 2008 2003 2004 0 642 626 16
r44 2008 2004 2005 0 642 626 16
r45 2008 2005 2006 0 642 626 16
r46 2008 2006 2007 0 642 626 16
r47 2008 2007 2008 0 642 626 16
r48 2008 2007 2009 0 642 626 16
r49 2008 2007 2010 0 642 626 16
r50 2008 2007 2011 0 612 596 16
r51 2008 2007 2012 0 602 586 16
r52 2008 2007 2013 0 602 586 16
r53 2008 2007 2014 0 602 586 16
r54 2008 2007 2015 0 584 568 16
r55 2008 2007 2016 0 584 568 16
r56 2008 2007 2017 0 546 530 16
r57 2008 2007 2018 0 546 530 16
r58 2008 2007 2019 0 546 530 16
r59 2008 2007 2020 0 546 530 16
r60 2008 2007 2021 0 546 530 16


Thank you very much. Kind regards,
Rattiya

[FernandoRios]

HI there,
1) the pretrend test is just a joint test that all "pre-treatment" effects are zero. In your case, you cannot reject the hypothesis
2) GAverage is, as the name suggest, the average ATT across all cohorts, weighted by their total size.
3) You cannot identify the exact number of observations by cohort, unless you have a perfect balance panel. and are using never treated as control.
Remember that each ATTGT (the results from the LOOONG table) is estimated separately, so it has its own sample. Those samples cannot be added up.
F

[Hideto Koizumi]

Dear Fernando,

Thank you very much for making these commands available!

I have a quick question. The help page of csdid says

Estimation Method

csdid is a generalization of drdid, and as such it allows for various estimators. It estimates every feasible 2x2 DiD design available in the selected sample.

In all cases, the earliest-period covariates are used for the estimation of the propensity score and outcome regressions. This is the base period for all post treatment ATTGTs and T for all pre-treatment ATTGTs.

To specify a particular syntax, one should use the option method(method) using one of the following key words:

It says the base period for all pre-treatment ATT is T rather than T-1. Did you actually mean T-1? Glancing at the ado file of drdid, I guess T-1 covariates are used for both outcome and propensity score models (line 1019 of drdid.ado, as follows), but I just wanted to make it sure.

            ** _delta
            bysort `touse' `ivar' (`tmt'):gen double `__dy__'=`y'[2]-`y'[1] if `touse'
            ** Reg for outcome 
 
            `isily' reg `__dy__' `xvar' if `touse' & `trt'==0  & `tmt'==0 [iw = `weight']

(y[2] - y[1] is the outcome, so I am guessing that `xvar' is in T-1 period)

I would very much appreciate your response.
Kindest,
Hideto

[FernandoRios]

Good catch
i will update the help file.
so it uses t and t-1 for all attgts as default when t<g
but uses t-1 g-1 when using the long option when t<g

[Hideto Koizumi]

Awesome, thank you for your prompt response Fernando!