How are the p values ​​of the synthetic control method obtained using only the "synth" command in STATA?

[Alexis Rodas]

Hi all,

I want to ask if anyone knows the following and can help me:

First, the topic is about the Synthetic Control Method.
Second, using the synthetic control method with the "synth_runner" command in STATA, the p-values are obtained directly; however, the time it takes to calculate them is a lot. So, I want to ask if anyone got the p-values (capable of a manual way) of the synthetic control method using the "synth" command and not the "synth_runner" command.

Regards,

AR

[Dimitriy V. Masterov]

synth_runner is actually running synth under the hood repeatedly, which is why this takes so long. The p-value calculation is described in section 2.2 of the Stata Journal paper. Essentially, you throw out the treated geos and repeat the synth analysis for each of the untreated units as if it was treated. You can also sample untreated units at random. If the distribution of placebo effects yields many effects as large as the main estimate, then it is likely that the estimated effect was observed by chance. The p-value is the proportion of control units that have an estimated effect at least as large as that of the treated unit.

This means there is not way to speed this up other than using the MP capacity of Stata to do this in parallel.

[Alexis Rodas]

Thanks Dimitriy,

I have an extra question, I have not been able to find an adequate explanation. When Synth_Runner is used, The command gives two types of results with respect to the p-values (pvals and pvals_std). Can you help me to know what is the difference between these two: pvals and pvals_std?

Alexis Rodas

[Dimitriy V. Masterov]

The false placebo effects may be quite large if those units where not matched well in the pre-treatment period. This would cause p-values to be too conservative. There are two solutions:

1) throw away any of the "false placebo" results if the pre-treatment match quality is below some threshold (the graph may still plot them however even if they are disregarded).
2) Divide all of the effects by the corresponding pre-treatment match quality. This is the standardization of the p-value.

This is all explained in section 2.2.