Propensity score matching - Balance diagnostics

[Rob Wood]

Hi,

I'm performing some propensity score matching and I want to assess how well my covariates are balanced after matching using the standard differences as proposed by Peter C. Austin in An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies. He suggests that the standard difference calculations are as follows for continuous and binary covariates, respectively:

However, as I am matching to obtain the Average treatment effect (ATE) after the matching process all (or nearly all) observations have two values for each covariate; the original covariates (treated or untreated) and the matched covariates (untreated or treated). Therefore, my question is, which arguments do I put into the above formulas?

Any help or advice would be greatly appreciated.

Thanks.

[Melissa Garrido]

Hi Rob,

To evaluate the balance of your covariates after matching, you'll want to use the means/proportions from your matched covariates.

If you use the user-written command -psmatch2- to do the matching, you can use another user-written command (-pstest-) to calculate the standardized differences for you.

Hope this helps,

Melissa

[Tom Weichle]

Hi Rob,
The standardized difference can be used to compare balance in measured variables between treated/untreated subjects in the matched sample with the unmatched sample. The user-written command -pstest- (which is part of the user-writter package -psmatch2-) can calculate several measures of the balancing of the variables before and after matching. Note: -pstest- uses variances from unmatched group in the matched group calculation. Typically, a standardized difference > 10% (in absolute value) represents meaningful imbalance in a given variable between treatment groups.

Tom

[Rob Wood]

Hi Tom and Melissa,

Thank you both for your feedback. I had discovered the user written command pstest, however if I am calculating the ATE via propensity score matching, it appears I have to perform pstest twice, once with the atu option and once without, as opposed to having only one set of balance diagnostics, is that correct?

Thanks.

[Melissa Garrido]

No, you should only need to run it the one time. The matches are created prior to calculation of either the ATE or ATT, so you just have the one set of matches to evaluate.

[Rob Wood]

Thank you again for your reply.

I was just confused by the following statement in the help file for pstest.

atu After psmatch2, ate one can specify this option to consider balancing for the untreated.