[Erik Nelson]

Hi @FernandoRios,

I am using csdid. I have a few questions about the csdid's output. Here is an example of my code:

csdid rpcinccpi distCBD if cty==1, ivar(StationID) time(year2) gvar(openingyear2) notyet long2
estat simple
estat calendar
estat group
estat event
csdid_plot, title("Event-Study")
estat pretrend

Is estat simple equation (3.10) in Callaway and Sant’Anna (2021)? Further, I assume that estat event generates the "Event study" line of estimates in Table 3 of Callaway and Sant’Anna (2021)?

Thanks for your help in advance!

-Erik

[FernandoRios]

1) Yes, that is the idea
2) No, the original paper CS(2021) produced "Dynamic" effectplot that is replicated if ou drop the long2 option. However, most people like the traditional event/like plot that is what you observe

[Anne Simon]

Dear Fernando,

I'm using the Stata csdid package as part of my job market paper. In the main specification, I cluster the standard errors at the district level (around 400 districts). As a robustness check, I, however, need to show that the results are robust when clustering at a more aggregate level (regions, 11 in total). For this purpose, I'm using wild bootstrap within the csdid command. I'm sharing the code that I'm using for this purpose below. I'm a bit surprised by how large the t statistics still is after wild bootstrapping standard errors. When I replicate the results with standard TWFE and wild bootstrap standard errors there, the p values get much larger - this may partly be due to smaller coefficients when using TWFE but I'm not sure the difference in how the statistical significance of the results changes with and without bootstrap standard errors should be so large. Could you please check the code to see if I'm applying the wild bootstrap command correctly? (I tried to closely follow the example in the Stata helpfile).

gen gvar = cond(reformyear== ., 0, reformyear) // group variable as required for the csdid command

csdid $outcome, time(birthyear) gvar(gvar) method(reg) agg(simple) cluster($se_distr)  

csdid $outcome, time(birthyear) gvar(gvar) method(reg) cluster($se_region) wboot rseed(1212) saverif(_rif_) replace 
    
use _rif_, clear

csdid_rif ATT  , wboot reps(2000) cluster($se_region) seed(1212)

Here are screenshots from the results window (1) after wild bootstrapping standard errors (I'm not showing the first part of the results here as the table is very large with lots of comparisons) and (2) with the more disaggregately clustered standard errors:

Thanks a lot in advance!

Kind regards,
Anne

[FernandoRios]

There is something else going on here.
Your results using csdid_rif and estat simple should be the same. Would need to double check what you are getting

[Anne Simon]

Thank you for your response! Is there anything you'd need from me to do that?

[FernandoRios]

Reproducible example

[Anne Simon]

Alright. I will share one with you on Monday/ Tuesday.

[Anne Simon]

Just a heads up:
I shared the data and code via email with you yesterday.

[FernandoRios]

Yes Saw that. Will take at least til friday to look at it

[aline zd]

Hello Fernando,

Thank you for your work on these packages. I am using csdid to conduct a heterogeneity analysis by subgroup. Specifically, I estimate ATT effects separately for males and females using a common treatment timing, with 4 years pre- and 2 years post-treatment. I'm working with panel data.

Specifically, I am running:

 csdid outcome  $controls if male == 1, ivar(id) time(year) gvar(first_treated) long method(reg)
 csdid outcome  $controls if male == 0, ivar(id) time(year) gvar(first_treated) long method(reg)

I would like to test whether the point estimates from these two subgroup models (e.g., the post-treatment ATT estimates) are statistically different. After running:

estat simple

I tried the following approach:

csdid outcome  $controls if male == 1, ivar(id) time(year) gvar(first_treated) long method(reg)
estat simple 
estimates store m1
   
csdid outcome  $controls if male == 0, ivar(id) time(year) gvar(first_treated) long method(reg)
estat simple
estimates store m2
     
suest m1 m2

However, I received the following error:

unable to generate scores for model m1
suest requires that predict allow the score option
r(322);

I then attempted:

csdid outcome  $controls if male == 1, ivar(id) time(year) gvar(first_treated) long method(reg)
estat simple 
predict att

But encountered another error:

variable t_2006_2009 not found
r(111);

I understand that csdid may not be fully compatible with suest or traditional predict commands. What would you recommend as the best way to formally test whether subgroup ATT estimates (from estat simple or estat event) are statistically different?

Is there a workaround or built-in approach in csdid that allows for this kind of subgroup comparison, accounting for uncertainty in the estimates?

I’m using csdid for my main analysis, so I’d like to use the same method for the subgroup analysis to keep things consistent.

Many thanks in advance for your help.

[FernandoRios]

You can’t do it with csdid
i posted some time ago something for this, which requires more steps.
but if you use jwdid instead, is far easier.

[FernandoRios]

You can’t do it with csdid
i posted some time ago something for this, which requires more steps.
but if you use jwdid instead, is far easier.

[Toby Low]

Dear Fernando,

I'm collaborating with Professor Giuseppe Moscelli on a research project for the NHS, and we have a question relating to the generation of confidence intervals for a not-yet-treated model specification.

I’m running csdid in Stata 17 on NHS data aggregated at the monthly provider level. Unfortunately I can't share the data as confidential.

Setup:

• Staggered treatment adoption, all providers eventually treated
• Control group specified as notyet (cannot use nevertreated due to lack of untreated units)
• Same issue affects almost all my outcome variables of interest

The issue:

• Coefficients and standard errors are identical irrespective of whether the model run uses pointwise or uniform confidence intervals (as expected)
• When we compute the Group ATT using the uniform and pointwise confidence intervals we generate the below results:

* Run uniform
csdid_stats group, wboot rseed(123456)


------------------------------------------------------------------------------
Group Coef. Std. Err. t [95% Conf. Interval]
------------------------------------------------------------------------------
GAverage 0.0089634 0.0043081 2.08 -2.24e+12 2.24e+12
G699 0.0219016 0.0057842 3.79 -3.00e+12 3.00e+12
G742 0.0076663 0.0051910 1.48 -2.70e+12 2.70e+12
G743 -0.0014223 0.0137789 -0.10 -7.16e+12 7.16e+12
G752 0.0099298 0.0027429 3.62 -1.42e+12 1.42e+12
G754 0.0274870 0.0018086 15.20 -9.40e+11 9.40e+11
G755 0.0062979 5.23e-18 1.2e+15 0.0035819 0.0090139
G756 -0.0068684 0 . -0.0068684 -0.0068684
------------------------------------------------------------------------------


Run pointwise
csdid_stats group, wboot rseed(123456) pointwise


------------------------------------------------------------------------------
Group Coefficient Std. Err. t [95% Conf. Interval]
------------------------------------------------------------------------------
GAverage 0.0089634 0.0043081 2.08 0.0006844 0.0172425
G699 0.0219016 0.0057842 3.79 0.0103802 0.0334229
G742 0.0076663 0.0051910 1.48 -0.0015862 0.0169187
G743 -0.0014223 0.0137789 -0.10 -0.0290579 0.0262133
G752 0.0099298 0.0027429 3.62 0.0039005 0.0159591
G754 0.0274870 0.0018086 15.20 0.0123555 0.0426186
G755 0.0062979 5.23e-18 1.2e+15 0.0035819 0.0090139
G756 -0.0068684 0 . . .
------------------------------------------------------------------------------

• For most of our units, the Uniform CI bounds are constant. This occurs whether we run using long2 or the default model option.

Questions:

1. Is this behaviour expected with notyet controls, perhaps due to sparse data for certain late adopting providers/no matching controls?
2. Would it be appropriate given the above for us to pivot and use the pointwise CIs?

Unfortunately I can’t share the dataset itself, but I can share the relevant code below:

Code
csdid "Outcome_Variable", ivar(group_id) time(year_month) gvar(group_var) notyet agg(event) long2 saverif(mod1) wboot replace rseed(123456) use mod1, clear
csdid_stats group, wboot rseed(123456)
csdid_stats group, wboot rseed(123456) pointwise

Any guidance or confirmation on whether this is expected behaviour (and how best to address it) would be greatly appreciated.

Kind regards,

Toby

[FernandoRios]

I think the problem has to do with cohort size.
when there is little variation, the distribution of the t-statistic (simulated) is huge. And, uniform confidence intervals take the largest of them all, which is why they are so large.
they may seem constant, because of their magnitud.

So you may need to decide if you have to this for every statistic and group or not

[Awais Farid Khan]

FernandoRios 1 - is difference between csdid and csdid2 outcomes expected?

2-i have question regarding method() option.....used ipw, stdipw, drimp and dripw.....the problem is results are different and highly inflated for drimp and dripw when covariates are added..is there any justification which method is appropriate? I will be obliged as CS paper is mainly focused on DRIPW and I want to justify scenarios not to use doubly robust estimators

country level data
year 2010 - 2020
Always=91
Later=28
Never=15

when method(ipw)

Difference-in-difference with Multiple Time Periods

Number of obs = 353
Outcome model :
Treatment model:
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
ATT | 4.517172 1.321312 3.42 0.001 1.927448 7.106897
------------------------------------------------------------------------------
Control: Not yet Treated



when method(stdipw)

Number of obs = 353
Outcome model : weighted mean
Treatment model: stabilized inverse probability
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
ATT | 4.917568 1.705818 2.88 0.004 1.574226 8.26091
------------------------------------------------------------------------------
Control: Not yet Treated



when method(drimp)

Number of obs = 369
Outcome model : weighted least squares
Treatment model: inverse probability tilting
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
ATT | 11.08214 2.971681 3.73 0.000 5.257754 16.90653
------------------------------------------------------------------------------
Control: Not yet Treated



when method(dripw)


Number of obs = 353
Outcome model : least squares
Treatment model: inverse probability
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
ATT | 19.43744 9.520719 2.04 0.041 .7771732 38.09771
------------------------------------------------------------------------------
Control: Not yet Treated