xtdidregress or xthdidregress?

[Thibaud Marcesse]

I am working with panel data on the implementation of a policy in rural India (2012-2021). I am trying to test whether the award of quotas of descriptive representation shapes the implementation of the policy. I am considering the award of a quota to be a “treatment,” which happens once and simultaneously to a group of villages (the rest remains untreated). I would think xtdidregress to be the way to go, but the command assumes stability of treatment effects. In this case, I am not sure the “treatment” effect does diminish or increase over time (there may be something about the quotas that shapes their effect overtime). Does this mean I should use xthdidregress? Xthdidregress appears better suited when there is more than one treatment, which is not the case with my dataset.

[George Ford]

if you suspect heterogeneity, then xthdidregress will give you the effect by year after the treatment.

Basic testing is available after either command. Or, you can get the same using xtreg/reghdfe with the full complement of tests.

I don't see anything that permits you to test homogeneity with xth. That would be handy, and maybe I've missed it. estat atetsplot may be helpful though not formal test.

One option would be the lrtest, but it's not legit due to the use of robust/cluster SE.

The AIC/BIC would be available after reghdfe/xtreg, but not xt(h)didregress

clear all

set obs 50
g id = _n
g treated = id>=6
g fe = runiform()
g x = rgamma(5,1)
expand 10
sort id
bys id: g year = _n

g post = year>=6
g did = post*treated

forv i = 6/10 {
    g did`i' = (year==`i')*treated
}

xtset id year

** HETERO EFFECTS
set seed 8568

g y = x + fe + (1*year)*treated*post + rnormal()

eststo a1: qui xtdidregress (y) (did) , group(id) time(year) vce(cluster id)
eststo a2: qui reghdfe y did , a(id year) cluster(id)
qui estat ic
qui estadd local aic = r(S)[1,5]
qui estadd local bic = r(S)[1,6]
eststo b1: qui xthdidregress twfe (y) (did) , group(id) vce(cluster id)
estat atetplot , name(plot_het, replace)
eststo b2: reghdfe y did6 did7 did8 did9 did10 , a(id year) cluster(id)
qui estat ic
qui estadd local aic = r(S)[1,5]
qui estadd local bic = r(S)[1,6]
esttab a1 a2 b1 b2 , keep(*did* 6* 7* 8* 9* 10*) stats(N aic bic)

** HOMO EFFECTS
set seed 8568
replace y = x + fe + 8*treated*post + rnormal()

eststo a1: qui xtdidregress (y) (did) , group(id) time(year) vce(cluster id)
eststo a2: qui reghdfe y did , a(id year) cluster(id)
qui estat ic
qui estadd local aic = r(S)[1,5]
qui estadd local bic = r(S)[1,6]
eststo b1: qui xthdidregress twfe (y) (did) , group(id) vce(cluster id)
estat atetplot , name(plot_hom, replace)
eststo b2: reghdfe y did6 did7 did8 did9 did10 , a(id year) cluster(id)
qui estat ic
qui estadd local aic = r(S)[1,5]
qui estadd local bic = r(S)[1,6]
esttab a1 a2 b1 b2 , keep(*did* 6* 7* 8* 9* 10*) stats(N aic bic)

[George Ford]

Added a joint test.

[CODE]
clear all

set obs 50
g id = _n
g treated = id>=6
g fe = runiform()
g x = rgamma(5,1)
expand 10
sort id
bys id: g year = _n

g post = year>=6
g did = post*treated

forv i = 6/10 {
g did`i' = (year==`i')*treated
}

xtset id year

** HETERO EFFECTS
set seed 8568

g y = x + fe + (1*year)*treated*post + rnormal()

eststo a1: qui xtdidregress (y) (did) , group(id) time(year) vce(cluster id)
eststo a2: qui reghdfe y did , a(id year) cluster(id)
qui estat ic
qui estadd local aic = r(S)[1,5]
qui estadd local bic = r(S)[1,6]
eststo b1: xthdidregress twfe (y) (did) , group(id) vce(cluster id)
qui test 6.year = 7.year = 8.year = 9.year = 10.year
local F : di %6.2f r(F)
qui estadd local F_joint = `F'
local p : di %6.4f r(p)
qui estadd local p_joint = `p'
estat atetplot , name(plot_het, replace)
eststo b2: reghdfe y did6 did7 did8 did9 did10 , a(id year) cluster(id)
qui estat ic
qui estadd local aic = r(S)[1,5]
qui estadd local bic = r(S)[1,6]
qui test did6 = did7 = did8 = did9 = did10
local F : di %6.2f r(F)
qui estadd local F_joint = `F'
local p : di %6.4f r(p)
qui estadd local p_joint = `p'
esttab a1 a2 b1 b2 , keep(*did* 6* 7* 8* 9* 10*) stats(N aic bic F_joint p_joint)

** HOMO EFFECTS
set seed 8568
replace y = x + fe + 8*treated*post + rnormal()

eststo a1: qui xtdidregress (y) (did) , group(id) time(year) vce(cluster id)
eststo a2: qui reghdfe y did , a(id year) cluster(id)
qui estat ic
qui estadd local aic = r(S)[1,5]
qui estadd local bic = r(S)[1,6]
eststo b1: xthdidregress twfe (y) (did) , group(id) vce(cluster id)
qui test 6.year = 7.year = 8.year = 9.year = 10.year
local F : di %6.2f r(F)
qui estadd local F_joint = `F'
local p : di %6.4f r(p)
qui estadd local p_joint = `p'
estat atetplot , name(plot_hom, replace)
eststo b2: reghdfe y did6 did7 did8 did9 did10 , a(id year) cluster(id)
qui estat ic
qui estadd local aic = r(S)[1,5]
qui estadd local bic = r(S)[1,6]
qui test did6 = did7 = did8 = did9 = did10
local F : di %6.2f r(F)
qui estadd local F_joint = `F'
local p : di %6.4f r(p)
qui estadd local p_joint = `p'
esttab a1 a2 b1 b2 , keep(*did* 6* 7* 8* 9* 10*) stats(N aic bic F_joint p_joint)
/CODE]

[Thibaud Marcesse]

Thank you so much, this perfectly answers my question George!

[Thibaud Marcesse]

One more thing I wanted to clarify For the first joint F test (heterogeneous treatments effects), the null is that there are heterogeneous treatment effects, and likewise for the second, the null is that treatment effects are homogenous over time, is that correct?

I ran the code and this was the result for the first F test, so wanted to make sure I did interpret this the right way:

----------------------------------------------------------------------------
N 144594 144594 144594 144594
aic . 666832.0 . 661654.7
bic . 666841.8 . 661704.1
F_joint 1208.24 1208.25
p_joint 0 0
----------------------------------------------------------------------------
t statistics in parentheses
* p<0.05, ** p<0.01, *** p<0.001

[George Ford]

The null is that they are all equal (homo).