Proper procedures for coarsened exact matching

[Mia Dana]

This is more of a method question than Stata-specific. Let me know if this isn't the right forum for it.

We’re using a natural experiment, where “treatment” occurred around mid-20th century and our outcome data comes from decades later. Selection into the treatment group wasn’t fully random, but we know what characteristics they used to select participants. We want to use coarsened exact matching (CEM) to reduce bias in comparing treatment and control. Our actual data is restricted-use, so I’ll use the example data from the -cem- article.


We have three areas where we’d like to clarify the proper way to implement CEM:

1. Can someone give more detail on how you decide which variables should be used in the actual matching vs. later used as controls in regressions? Put differently, do you lose the balance you create by matching when using controls?

Our current approach is to use the variables that we know influenced selection into the treatment group as the variables to match on and then use any remaining variables that we think are relevant (or ones that would result in us losing too many observations if we used it in the actual matching) as covariates in the regressions. For example:

use http://www.mattblackwell.org/files/stata/data/lalonde.dta, clear

/* participants were selected into treatment based on initial income, initial employment status,
and household structure so we match on those variables: */
cem re74 (#4) u74 married, treat(treated)
* then we'd run a basic regression with other controls, e.g. education:
reg re78 treated nodegree [iweight = cem_weights]

2. You need to rerun the -cem- command each time you change the sample, correct? For example, we’d like to rerun our main analysis on subsamples, such as by race. After running the above code, we do this:

cem re74 (#4) u74 married if black == 1, treat(treated)

reg re78 treated nodegree [iweight = cem_weights] if black == 1

3. We also plan to use inverse probability weighting (IPW) to account for sample issues, like people who didn’t live long enough to be observed in our outcome data, since there’s a gap of several decades between treatment and outcome. If we use CEM and IPW together, would it a problem to use the same variables for CEM and to create the IP weights?

[George Ford]

In my experience, if you cem on a few variables, you can then get imbalance for the excluded ones even if they were balanced before.

I usually start by looking at covariance balance. (covbal does a nice job). If the |Stan Diff| < 0.25 for everything (imbens/wooldridge), you don't need to bother (unless you use a smaller threshold).

Start with the really bad ones, but then covbal again with the weight to see if it messed up the balance on the X excluded from cem.

You'll probably need to include most Xs.

It can take some fiddling with how you balance (scott, ranges, etc...) to get it right.

As for subsamples, you may (or may not) want to do it again. If the Xs are balanced within the chosen "if" group, then you may not need to balance at all. But, from the full cem, the Xs should be balanced for the subsample (but you may get more observations from doing it again).

I've mostly switched to ebalance from cem. Faster and ebalance can target more moments.

[Mia Dana]

Thank you!