In space placebo error: variable does not exist

[Harina Peternella]

Hi all,


First of all, the best wishes for 2023!


I am struggling with my last placebo test (in-space) for my research and I have really no clue what is going wrong.
I research the effect of treatment of one country on 3 different transport sectors using the synthetic control. Right now I want to perform the in-space placebo test for the last sector.
I use the same command as I did before:

allsynth air_tkm lag_air_tkm air_departures gdp_capita population emp(1981(1)2008) pl_x(1970(1)2008) pl_m(1970(1)2008), trunit(70) trperiod(2004) unitnames(country) keep(placebo1c.dta, replace) pvalues gapfigure(classic lineback, save(air)) nested allopt

However, now I get this weird error which confuses me, it says: air_depa_X2 does not exist as a (numeric) variable in dataset (r198)

If I drop air_departures the code does run! If I change the position of air_departures in the code it changes the error in air_depa_X1 or air_depa_X3 does not exist.
The format is (double) %10.0g, which is the same as for instance gap_capita. So, this should not be the problem as well. I cannot find any missing variables. I really don't understand why it does not run.
If I do the normal code again (so without the "values gapfigure(classic linebacker, save(air))" it does run.

Did someone encounter the same error and knows how to solve it? Or does anyone know how I can figure out what goes wrong? Like I said, I am clueless at the moment.
For the other sectors I could run the code without any problem.


Thank you in advance!

[Harina Peternella]

Okay, I kept trying (now I am triggered haha) and with the "set trace on" command I found the following possible explanation for the error:

However, I am missing way too many skills to understand what is going on. If someone knows, I would be so grateful.

[edit: sorry the picture did not upload]

[Jared Greathouse]

I have many comments on this. First, please either provide the link your dataset came from or give an example of it with dataex.


EDIT: And please, for the love of God, no pictures. Please.... please, read the FAQ, we beseech you to not upload pictures pretty much ever. I'm not trying to be mean, it's just that pictures don't help us, help you.


Second edit: I don't think you know what you're doing. You're treatment is in 2004. But your lags go into 2008. That's not how it works!!!! You're training on test data, as we'd say in machine learning. This is the exact opposite way to use SCM, your results will ALWAYS be totally wrong like that! You'll ruin all your effect sizes this way and produce the wrong results.


I think code aside, there are way more fundamental issues at work. I can give you great resources on SCM, if you'd like.

[Harina Peternella]

Jared Greathouse Thank you for your reply.
For sure, I am not an expert on SCM or econometrics/machine learning in general.

Rereading the Abadie et al. (2010) paper I realize my mistake with the lags. It should have been a selection of lags (pre-treatment), for instance air_tkm(1975) and air_tkm(1987).
Thank you for noticing. I will reread some of the literature again. If you have some other resources on SCM I would be happy for that as well. I read your paper on SCUL too, and I really liked it. That is why I am considering to use SCUL as an additional robustness analysis.


Moreover, I included an example of my data below, because I don't think the lag problem will solve the issue with the air_departure variable when running the in-space placebo.

* Example generated by -dataex-. For more info, type help dataex
clear
input byte country_id str78 country int year byte treatment double(air_tkm air_departures gdp)
2 "Algeria"   1970 0     3.1  14800 2368574659317178
2 "Algeria"   1971 0     3.7  17400 2042663363627491
2 "Algeria"   1972 0     4.2  19300 2532392779232273
2 "Algeria"   1973 0     5.2  26200 2558030158653868
2 "Algeria"   1974 0     6.6  29300 2674970435617106
2 "Algeria"   1975 0     7.2  31900 2732324364115853
2 "Algeria"   1976 0     9.1  34400 2878610001141668
2 "Algeria"   1977 0       9  32900 2944321111579685
2 "Algeria"   1978 0    11.2  35000 3123376575801853
2 "Algeria"   1979 0    12.3  34800 3258731214586459
2 "Algeria"   1980 0    12.9  36500 3186437164465725
2 "Algeria"   1981 0    14.2  38200  318225985327508
2 "Algeria"   1982 0    15.9  38700 3281860856022247
2 "Algeria"   1983 0    28.2  40300 3352699713845005
2 "Algeria"   1984 0    14.9  43400 3432806399482912
2 "Algeria"   1985 0      21  46200 3453821596873789
2 "Algeria"   1986 0     7.7  50500 3366938886625954
2 "Algeria"   1987 0    13.3  48400  324888117439136
2 "Algeria"   1988 0    13.9  49400 3128337532949908
2 "Algeria"   1989 0    23.1  45700 3179773320648016
2 "Algeria"   1990 0    14.6  44100 3123997664333553
2 "Algeria"   1991 0    24.3  45600 3011499933301611
2 "Algeria"   1992 0    20.2  45900 2994491320383532
2 "Algeria"   1993 0    21.3  45400 2867195894496503
2 "Algeria"   1994 0    24.2  45500  278315503976743
2 "Algeria"   1995 0      21  47000 2834265192854013
2 "Algeria"   1996 0      16  45100 2899193201640835
2 "Algeria"   1997 0    16.8  49600 2884120227978207
2 "Algeria"   1998 0    18.8  44200 2986055665596795
2 "Algeria"   1999 0    15.4  36400   30382729153868
2 "Algeria"   2000 0  12.647  40202 3111176168980686
2 "Algeria"   2001 0  18.346  49274 3162811507050648
2 "Algeria"   2002 0   17.89  46015 3297612811114808
2 "Algeria"   2003 0  19.091  44315 3490223247167668
2 "Algeria"   2004 0   21.44  48531 3592645301057895
2 "Algeria"   2005 0  31.624  45692  375209616063379
2 "Algeria"   2006 0  23.694  44822  376015536841842
2 "Algeria"   2007 0  16.572  44136 3828148476291098
2 "Algeria"   2008 0  16.985  31026 3856407659501647
5 "Argentina" 1970 0    47.9  71700 9243256579255088
5 "Argentina" 1971 0    57.4  75600 9613676881153724
5 "Argentina" 1972 0    79.7  79000 9614237930780166
5 "Argentina" 1973 0    85.9  71100  972546704467513
5 "Argentina" 1974 0    98.2  79000 1010006847865704
5 "Argentina" 1975 0    74.6  89800 9939696623293234
5 "Argentina" 1976 0    96.3  90000 9591172000948284
5 "Argentina" 1977 0   119.8  93600  101036273664291
5 "Argentina" 1978 0   120.5  96500 9505856210168488
5 "Argentina" 1979 0   136.8 103700  103212391038659
end

[Jared Greathouse]

This is the paper where SC just began to make complete sense to me. This chapter does an amazing job too.

Note I'm biased, but I would skip normal SCM altogether, since scul outperforms it. I can guide you through it if you'd like