Estimating treatment effect with synthetic control method

[Eran Itskovich]

Hi everyone,

A beginner's question regarding the synthetic control method:

I want to estimate the effect of an intervention program, and to this end, I implement the synthetic control method using the -synth- command.
After running the command, I get the weights of each comparison unit (from the donor pool) and a graph showing the pre- and post-treatment trends.
However, it is not fully clear to me how to check whether there's a significant difference between the treatment unit and the synthetic control, before and after the intervention. Should I just -keep- the results and then perform a distinct analysis (e.g., difference-in-differences)? or is there a test implemented in the -synth- command?

Thanks!

[George Ford]

there are many alternatives to synth (synth_runner, for example). These often do the "testing" for you, though sometimes require you to ask for it. I've seen a few newer programs that will do it, but am not familiar with them.

[Jared Greathouse]

Much I could say, but in a minute (in like, an hour or two) I'll sort of go into inference with SCM. It's still kind of an open topic in the literature, and there's no agreed on way to do this across all estimators (even with normal SCM). My INITIAL thoughts though is that lots of the ways in Stata kind of aren't as good as the ones in R or Python (all Stata's are p value based, such as synth2). So, I'll talk able the more classical methods to do this, when I've got the time.

[Jeff Wooldridge]

The user-contributed command sdid reports an estimated treatment effect in the post treatment period, and reports a standard error, too. Jared knows much more about the quality of the inference. I've played around with a few data sets with a single treated unit, and the reported precision strikes me as suspiciously good in some cases. The caveat -- given with the command -- is that the standard errors are based on asymptotic approximations. So they're likely to be suspect without a lot of time periods, and maybe with strongly persistent time series. But you will get an estimate and standard error.

[Eran Itskovich]

Many thanks!

[Dimitriy V. Masterov]

synth_runner leaves behind K p-values for the two-sided hypothesis that the effect is zero in each of K post-treatment periods. These are stored as locals. There is also a single p-value that all K effects are jointly zero. Take a look at the SJ paper for some code examples.

[stefan tanevski]

Dear Statalist Members,

I am a beginner in this area and have been grappling with the correct methodological approach for my study. I would greatly appreciate any insights from the community.
More context below.


Background & Research Objective

I am studying the potential liquidity and economic benefits of a hypothetical Western Balkan stock exchange merger. Since no actual merger has taken place, standard difference-in-differences (DiD) or event-study methodologies do not apply due to the absence of an observable treatment group.

Given this, I am considering using the Synthetic Control Method (SCM) to construct a counterfactual comparison. My approach involves creating two separate synthetic control groups:

1. First synthetic control (Non-Merged WBSE): Construct a counterfactual Western Balkan stock exchange that represents the region’s stock market in the absence of a merger. The donor pool consists of the Baltic states (Estonia, Latvia, Lithuania) before their integration into NASDAQ OMX, selected for their historical similarities with the Western Balkans in terms of market size, transition from socialist economies, and financial integration processes.

2. Second synthetic control (Hypothetical Merged WBSE): Construct a synthetic version of a merged Western Balkan stock exchange. The donor pool remains the same, but the weighting scheme would be adjusted to reflect expected post-merger liquidity and efficiency improvements.


Challenges & Questions

• Is SCM appropriate for this setup? SCM is typically used to estimate a counterfactual for an actual intervention, whereas here, both the “treated” and “control” units are synthetic constructs. Would a modified version of SCM still be valid?

• Weight adjustment for the merged exchange: Standard SCM optimizes weights based on pre-treatment characteristics, but I am proposing an adjustment to reflect expected post-merger improvements. Is there an established way to handle such cases in SCM?

• Implementation in Stata: Given that I am constructing two synthetic controls, should I run separate SCM estimations for each case, or is there an alternative way to compare them directly within the same framework?

• Alternative methodologies: If SCM is not the best-suited approach, what alternative econometric techniques would you recommend for evaluating a hypothetical merger?