Hi all,
I have a fully balanced panel data with al US states that covers two years but aggregated on monthly level (24 time points in total). I am trying to analyze the impact of a policy change that affected only one state, therefore I have only one treated unit. The treatment started on July 2010, meaning that I have 18 months of pre-treatment period and 6 months of post-treatment period.
My first approach was using Synthetic DID and I found the ATT to be statistically not significant (using placebo method to calculate the standard errors as that is the only option for my case). Then I tried doing Abadie-style Synthetic Control (will refer to it as ADH method from now on). However, instead of trying to match on other covariates, I only used lagged outcomes to create the synthetic unit (let's call it constrained regression). In addition, I did not assign equal importance weight to each lagged outcome but let the nested optimization process to find the optimal covariate weights. As you can see, I am trying to mimic the SDID logic as I calculate time weights in principal.
Now, comparing the insignificant ATT that SDID gives me with my version of SC, (after doing the classical in-time placebo, in space placebo inference tests and leave-one-out robustness check), I feel like the SC performs better. Why do I think that?
1- I compare the time weights that SDID gives and covariate weights that SC gives (that is similar to time weights, or you might call it the same), and the time weights of SC is much more intuitive than SDID's. While SDID gives very high weights to common shocks (0.61 to April 2009, followed by September 2009 with a weight of 0.23), SC gives the highest weights to July-December 2009 (all pre-treatment months get a weight, lowest is around 0.03 and highest is around 0.1).
2- I compare the state weights that SDID and SC gives. While SDID gives almost all of the states some weight, SC only gives to around 10 states, and the state that gets the highest weight is a state that is pretty similar to my treated state.
I hope the point I'm trying to make is clear because I wanted to ask opinion about this issue. I can provide further details if needed. Thank you in advance.
Serkan
p.s. this is my first post in the forum, I hope I am not breaking any forum rules. The packages I am using are sdid and synth2, which are very simple to use.
I have a fully balanced panel data with al US states that covers two years but aggregated on monthly level (24 time points in total). I am trying to analyze the impact of a policy change that affected only one state, therefore I have only one treated unit. The treatment started on July 2010, meaning that I have 18 months of pre-treatment period and 6 months of post-treatment period.
My first approach was using Synthetic DID and I found the ATT to be statistically not significant (using placebo method to calculate the standard errors as that is the only option for my case). Then I tried doing Abadie-style Synthetic Control (will refer to it as ADH method from now on). However, instead of trying to match on other covariates, I only used lagged outcomes to create the synthetic unit (let's call it constrained regression). In addition, I did not assign equal importance weight to each lagged outcome but let the nested optimization process to find the optimal covariate weights. As you can see, I am trying to mimic the SDID logic as I calculate time weights in principal.
Now, comparing the insignificant ATT that SDID gives me with my version of SC, (after doing the classical in-time placebo, in space placebo inference tests and leave-one-out robustness check), I feel like the SC performs better. Why do I think that?
1- I compare the time weights that SDID gives and covariate weights that SC gives (that is similar to time weights, or you might call it the same), and the time weights of SC is much more intuitive than SDID's. While SDID gives very high weights to common shocks (0.61 to April 2009, followed by September 2009 with a weight of 0.23), SC gives the highest weights to July-December 2009 (all pre-treatment months get a weight, lowest is around 0.03 and highest is around 0.1).
2- I compare the state weights that SDID and SC gives. While SDID gives almost all of the states some weight, SC only gives to around 10 states, and the state that gets the highest weight is a state that is pretty similar to my treated state.
I hope the point I'm trying to make is clear because I wanted to ask opinion about this issue. I can provide further details if needed. Thank you in advance.
Serkan
p.s. this is my first post in the forum, I hope I am not breaking any forum rules. The packages I am using are sdid and synth2, which are very simple to use.
