PSM Caliper 1:many without replacement?

[Helen So]

Hi all,

I'm trying to run a 1:many propensity score matching using "psmatch2", I was trying the "noreplacement" option and it said "Matching without replacement is only implemented with 1-to-1 propensity score matching". Is it possible to do 1:many matching without replacement in Stata?

Thanks in advance.

Helen

[Clyde Schechter]

Yes, it can be done. You cannot do it with -psmatch2-. But once you have calculated the propensity scores themselves, you can write code that will do a 1:many match without replacement. The general approach is illustrated in https://www.statalist.org/forums/for...d-total-assets, especially at #12.

That said, there is a reason that the readily available packages don't offer this option. The reason is that matching without replacement is a bad idea, and in the context of 1:many matching it is even worse. First, let's be very clear: there is absolutely zero statistical advantage to using matching without replacement. It offers no statistical advantages over matching with replacement. The preference that many people have for matching without replacement is based purely on aesthetics, or to put it less charitably, "vibes."

On the other side, matching without replacement has a serious disadvantage. In any matching procedure, there is always the potential problem that some observations in the data will fail to find a suitable match in the data. Sometimes this is a minor problem, and sometimes it is serious. That all depends on the data itself. The consequent exclusion of unmatchable observations from the data is bad in that it both reduces your effective sample size and power, and it may introduce bias into the analysis. In fact, incomplete matching is a reason why matching is sometimes avoided altogether.

If you match without replacement, whatever the severity of the failure to match problem would have been with replacement, in matching without replacement it is, at best, equally bad, and often it is worse, sometimes much worse. The reason is that in matching with replacement, failure to match only happens to observations that have unusual values of the match variable(s). But in matching without replacement, an observation that is perfectly matchable may fail to match because other observations have already taken up its potential matches, so it is left "orphaned." When you are doing 1:1 matching, this drawback of matching without replacement is bad enough. With many to 1 matching, it is even worse, because the available matches get eliminated from the match pool even faster.

If you are working with a data set that has a very large number of observations on the m-side of 1:m matching, and a relatively small number on the 1-side, then things might go well. (Of course, with a very large data set, don't complain if it takes a long time to run, and be prepared for the possibility that you will exhaust memory limits while trying to run it.) But otherwise, the probability is high that 1:m matching without replacement will end unhappily.

If you are truly wedded to the idea of matching without replacement, or if you are under orders from a supervisor who wants it done that way, then go ahead and try the approach I linked to above. But otherwise, I really recommend you go with matching with replacement.

[Helen So]

Thanks for suggestion. Yeah we do have a lot of controls relative to the treatment group (like 3M vs. 150), but you are right about the memory limits. I'm not sure if it's worthwhile putting in the efforts to get a worse match.