Placebo test for model with fixed-effects

[Khati Zolfaghari]

Dears,

I have a question about the model specification that I wonder if people can share their thoughts about it. I estimate a model using a panel dataset including 144 countries.

𝑦𝑖,𝑡+𝑘 = 𝛼𝑖𝑘 + 𝛾𝑡𝑘 + 𝛽𝑘𝐷𝑖,𝑡 + 𝜃𝑘𝑋𝑖,𝑡 + 𝜀𝑖,𝑡+𝑘

where

where 𝑦𝑖,𝑡 is the fertility rate for country 𝑖 in year 𝑡; 𝛼𝑖 are country fixed effects, included to
take account of differences in countries’ average fertility; 𝛾𝑡 are time fixed effects, including
to take account of global shocks and global business cycle; 𝐷𝑖,𝑡 is a dummy variable
indicating a pandemic recession event in country 𝑖 at year t. 𝑋𝑖,𝑡 is a vector that includes two
lags of the dependent variable and the pandemic recession dummy

I am interested to do a placebo test, but unfortunately, as I am searching but I do not know how can I do this.


any comments about this will be appreciated!
Thanks,

[Fei Wang]

You may use -ritest- (from SSC) with -cluster(countryID)- option to shuffle values of D_it at the country level for the placebo test. As you include lagged terms of D_it, please use lag operator (L.D, L2.D, etc.) so that -ritest- would randomize all terms related to D_it. The Stata Journal article is here.

[Khati Zolfaghari]

@Fei Wang thank you so much for your reply. I read the Stata Journal article, and I did it in this way as you recommended. But I do not know why there is not any change in my results related to the past.

ritest dum_recession _b[dum_recession], cluster(ifscode)  ///
local hmax = 5

cap drop b u d Years Zero
gen Years =_n-1 if _n<=`hmax'
gen Zero = 0   if _n <=`hmax'
gen b=0
gen u=0
gen d=0
forv h = 0/`hmax' {

reghdfe lnfertility`h' l(0/2)dum_recession l(1/2)lnfertility , absorb( i.ifscode i.year) vce(cluster ifscode)
replace b = _b[dum_recession]                    if _n == `h'+1
replace u = _b[dum_recession] + 1.645* _se[dum_recession]  if _n == `h'+1
replace d = _b[dum_recession] - 1.645* _se[dum_recession]  if _n == `h'+1
 
}

I think in my code, there is a problem. I am so thankful to receive your assistance and recommendation.

Best regards,

[Fei Wang]

I didn't get your code, frankly. You may need a one-line code like

ritest dum_recession _b[dum_recession], cluster(ifscode): reghdfe ....

[Khati Zolfaghari]

@Fei Wang thank you so much for your reply. In my code, I want to estimate the effect of pandemic recession(which takes value 1 or 0) on the fertility rate. so: the dependent variable is fertility, the explanatory variable is a dummy variable indicating the pandemic recession t and as the control variable, I add the two lags of fertility rate and two lag of pandemic recession.

I want to estimate the impulse response function for time horizon=5 years.

so based on this explanation how should I do for the placebo test and impulse response function?

when I run it I received this error:

ritest dum_recession _b[dum_recession], cluster(ifscode) :reghdfe f(0).lnfertility l(0/2)dum_recession l(1/2)lnfertility , absorb( i.ifscode i.year) vce(cluster ifscode)

dum_recession does not seem to be constant within clusters
r(9999);

Many thanks in advance for your valuable time and advice.

best regards,

[Rich Goldstein]

the use of -ritest- does not meet what I think of as a "placebo" test - for me, a placebo test means using a different outcome variable that is carefully chosen so that you do not expect the treatment/exposure to have an effect; I am not in your field and don't know what data you have access to so I can't give direct advice

[Khati Zolfaghari]

@Rich Goldstein thank you so much for your reply.

through my estimation:
the placebo test is conducted estimating the impulse response attributing the values of measure of shock__Di,t__ randomly , across the whole sample.

so here my shock is dummy variable which takes 1 in the year of pandemic recession and 0 othewise.
I use a pannel data among 144 countries from 1996-2019
my dependent variable is fertility rate .

many thanks for your assistance.
best regards,

[Fei Wang]

@Fei Wang thank you so much for your reply. In my code, I want to estimate the effect of pandemic recession(which takes value 1 or 0) on the fertility rate. so: the dependent variable is fertility, the explanatory variable is a dummy variable indicating the pandemic recession t and as the control variable, I add the two lags of fertility rate and two lag of pandemic recession.

I want to estimate the impulse response function for time horizon=5 years.

so based on this explanation how should I do for the placebo test and impulse response function?

when I run it I received this error:

ritest dum_recession _b[dum_recession], cluster(ifscode) :reghdfe f(0).lnfertility l(0/2)dum_recession l(1/2)lnfertility , absorb( i.ifscode i.year) vce(cluster ifscode)

Many thanks in advance for your valuable time and advice.

best regards,

Sorry that my suggestion was a bit misleading as I thought -ritest- would shuffle the whole set of values of a cluster but it can only do so when values are constant within the cluster. My idea: for example, all countries have data in years 2010-2020. Country A had pandemic in 2015-2017, and country B had pandemic in 2017-2019. A shuffle on D_it could switch the entire pandemic process between A and B, and placebo tests are passed when estimates are insignificant with shuffled data.

If you drop the -cluster()- option, then I think Stata is technically able to run the code. But it will shuffle all 1s and 0s of D_it through the whole data. After randomization, country A could had pandemic in 2011, 2013-2015, and 2019, and similarly for other countries pandemic would be irregularly scattered to different years. If you are able to accept such randomization for placebo tests, then dropping -cluster()- would do.

If you wish to have seemingly reasonable pandemic period even after randomization, then coding solution could be a bit complicated depending on your data structure. Have all countries been experienced pandemic? For countries having pandemic, did it start/end in the same year or different years? Solutions may vary.

Thanks for Rich's advice. In my field (economics), placebo tests usually mean that insignificant effects would be expected when treatment interventions are randomly reassigned (consistent with your ideas -- we shouldn't observe significant effects when there really are no effects).

[Khati Zolfaghari]

@ Fei Wang thank you so much for your reply.
in my data ,all countries are not experienced the pandemic.
also the year of pandemic is not the same for all of them .
some of them in the year of 200-2005-2009
some 2016-
some 2015 -2019

many thanks in advance for your valuable time and advice
best regards,

[Fei Wang]

I constructed an example data as below: 25 subjects in years 2001-2004; y is the dependent variable, and d is the event indicator. Codes in red may be replaced by your own case.

I define a program named "placebo" to conduct the placebo test by randomly assigning starting and ending years of the event to each subject. Then simulate the process for 200 times and plot all the simulated coefficients. You may see from the graph that all simulated (fake) coefficients should be around zero and substantially differ from the true coefficient marked by a red vertical line.

* Example generated by -dataex-. For more info, type help dataex
clear
input float(id year d y)
 1 2001 0   -.6908034
 1 2002 0   1.0937159
 1 2003 1    1.775657
 1 2004 1    3.068176
 2 2001 0   -.8758152
 2 2002 1    .8085946
 2 2003 0   .13539873
 2 2004 0    1.246992
 3 2001 0  -.11754683
 3 2002 0   1.0980148
 3 2003 0   2.7997336
 3 2004 1    2.391498
 4 2001 0 -.031690795
 4 2002 0   -.9599909
 4 2003 0   -.3647522
 4 2004 1    .7262428
 5 2001 0   -2.729693
 5 2002 1  -1.0273612
 5 2003 1    .5498356
 5 2004 0   -1.002292
 6 2001 0  -1.0595098
 6 2002 1  -.21475387
 6 2003 0  -.01952432
 6 2004 0   -.2606025
 7 2001 0    .7586559
 7 2002 0   -.6185099
 7 2003 1    2.898661
 7 2004 1    1.243234
 8 2001 0   -.6005324
 8 2002 0  -2.2300794
 8 2003 1  -1.0842277
 8 2004 0  -1.1408689
 9 2001 0    .7503044
 9 2002 0    .7674941
 9 2003 0    2.530894
 9 2004 1    2.587614
10 2001 0  -1.5734034
10 2002 0  -4.1846685
10 2003 0   -.4783316
10 2004 1 -.031441286
11 2001 0    .7563403
11 2002 1   1.1608492
11 2003 0   1.0109624
11 2004 0    3.718587
12 2001 0  -1.9011856
12 2002 0   -1.730064
12 2003 0   1.1471612
12 2004 1    1.918502
13 2001 0  -2.3461463
13 2002 0   -2.728561
13 2003 1   -.2695483
13 2004 0   -3.673393
14 2001 0   -.4416822
14 2002 1   .06229204
14 2003 1    .4700436
14 2004 1   2.0574238
15 2001 0   -1.886268
15 2002 0  -1.8185896
15 2003 1  -1.1438677
15 2004 1   -.7400825
16 2001 0   1.4635546
16 2002 0  -1.4228075
16 2003 0   1.6258308
16 2004 1    .3189145
17 2001 0  -1.8807546
17 2002 0    .7050123
17 2003 1   1.0133798
17 2004 1   -.4849722
18 2001 0  -1.0862764
18 2002 0  -.08783266
18 2003 0    .5446146
18 2004 1    .4737375
19 2001 0  -2.1588287
19 2002 0    .6472781
19 2003 1    .9010229
19 2004 0   -.5268855
20 2001 0    .1525243
20 2002 1    -.840714
20 2003 1    1.928343
20 2004 1    .8040664
21 2001 0    .4778997
21 2002 0   -.8524632
21 2003 0   .04877011
21 2004 1    2.391441
22 2001 0   -.8148755
22 2002 1   2.3420897
22 2003 0    1.750802
22 2004 0    .7683991
23 2001 0   -4.041288
23 2002 0  -2.8630886
23 2003 1   -1.837585
23 2004 1   -2.828412
24 2001 0    .3401876
24 2002 1  -1.1985407
24 2003 0   1.2037877
24 2004 0    .3608665
25 2001 0    .9204417
25 2002 1    2.226554
25 2003 1   1.6162157
25 2004 1   3.6385524
end

reghdfe y d, absorb(id year) vce(cl id)
local coef: disp %5.2f _b[d]

cap program drop placebo
program define placebo
    args sy ey
    preserve
    bys id: gen sy = runiformint(`sy',`ey') if _n == 1
    bys id: gen ey = runiformint(sy,`ey') if _n == 1
    bys id (sy): replace sy = sy[_n-1] if mi(sy)
    bys id (ey): replace ey = ey[_n-1] if mi(ey)
    replace d = inrange(year,sy,ey)
    reghdfe y d, absorb(id year) vce(cl id)
    restore
end

simulate _b[d], reps(200): placebo 2001 2004  // 2001 and 2004 are the earliest and latest possible starting years of the event.

sum _sim_1
local min = -max(abs(r(min)), abs(r(max)), abs(`coef'))
local max = -`min'
twoway kdensity _sim_1, ///
       xline(`coef') xlab(0 `coef') range(`min' `max') ///
       xtitle("Coefficients from placebo regressions") ytitle("Density")

[Khati Zolfaghari]

@Fei Wang thank you so much for your reply. I try to do .

best regards ,

[SHUHUI WANG]

@Fei Wang

thank you for your suggested code.
I am wondering if it is possibe to save these coefficients as the Distribution table, including Mean , 1% , 5%, 10%, 25%, 50% and 90%, 99% value of coefficients and also their t-statistic. Thank you a lot.