Calculating covariate means AFTER propensity score matching

[degruyterc]

Hi, I'm fairly new to Stata and am using version 13 for Windows. I have been conducting propensity score matching using teffects psmatch with nearest neighbour (1, 3 and 5). My question is hopefully fairly simple:

Prior to matching I have the means of my covariates, with and without treatment. To ensure the matching process was adequate, how can I obtain the means of my covariates with and without treatment AFTER matching? Furthermore, as I would like to conduct t-tests to ensure the means are not significantly different for each covariate, how I can obtain the standard deviation and number of observations for each covariate after matching?

Thanks for your help.
Chris

[Philip Jones]

Hi Chris,

One recommendation would be to use a hybrid approach: using -teffects psmatch- for the benefit of its improved standard errors (see here for more info) when you are estimating effect size, and use -psmatch2- for the easy way to display standardized differences pre- and post-matching.

An Example:

webuse cattaneo2
logit mbsmoke i.mmarried c.mage c.medu c.nprenatal
predict ps, pr

psmatch2 mbsmoke, outcome(bweight) p(ps) caliper(0.2) neighbor(1) noreplacement common

​pstest mmarried mage medu nprenatal, both treated(_treated) graph label
psgraph, treated(_treated) pscore(_pscore) support(_support)

The last two commands will give you some diagnostics about your matching. The -pstest- command in particular will give you the means of covariates, standardized differences, and reductions in bias after matching, which is what I think you are looking for. Also, the t-tests are included in its output.

Be sure to read the link I included above: further down on that page it is mentioned how to get your -teffects psmatch- and -psmatch2- commands to get the same output.

I hope this helps,

Phil

[Clyde Schechter]

As an aside, I notice that questions like this, where people are looking for a way to directly access the matching done in -teffects- seem to come up fairly often. Wishlist for Stata 14: make this information available after -teffects- either through variables left behind in the data set, or through follow-on commands.

[Philip Jones]

Very good point Clyde. Agree completely. Hopefully this will be added!

[degruyterc]

Thanks for your reply Philip. This is exactly what I was after - much appreciated.
And yes, I certainly agree with Clyde's comment too.

[Melissa Garrido]

Rather than using t-tests to evaluate balance, I'd suggest using standardized differences. T-tests are sensitive to sample size (and might miss covariate imbalance) and should be used for hypothesis testing where you want to generalize to a larger population – standardized differences work well when examining characteristics of a given sample

[Richard Williams]

I just came across this page today, in case it helps.

http://www.ssc.wisc.edu/sscc/pubs/stata_psmatch.htm

[Philip Jones]

Yes, that is the page I linked to in my post. However, I must say, sometimes formatted links don't show up as obviously as full URLs!

[daniel klein]

At this years German Users Group Meeting we had a chance to discuss exactly this issue with David Drukker. He explained that t-tests are not valid after the matching, because the tests do not take into account the estimated nature of the balancing process (i.e. the first step). Instead they take the data as if they were observed, and this seems not be theoretically sound. Unfortunately, I cannot elaborate more on this, but can say, that Stata Corp did not forget to include this feature, but omitted it for (good) reason. Therefore I think it is highly unlikely that this will be added in the future.

Best
Daniel

[Richard Williams]

Yes, that is the page I linked to in my post. However, I must say, sometimes formatted links don't show up as obviously as full URLs!

Whoops. Yes, I don't use formatted links, because I personally often miss the fact that they are links. Although your "see here for more info" would have been a clue.

[Philip Jones]

At this years German Users Group Meeting we had a chance to discuss exactly this issue with David Drukker. He explained that t-tests are not valid after the matching, because the tests do not take into account the estimated nature of the balancing process (i.e. the first step). Instead they take the data as if they were observed, and this seems not be theoretically sound. Unfortunately, I cannot elaborate more on this, but can say, that Stata Corp did not forget to include this feature, but omitted it for (good) reason. Therefore I think it is highly unlikely that this will be added in the future.

Best
Daniel

True, but they could automate the nice display of standardized differences pre- and post-matching.

[Jesse Wursten]

Sorry for resuscitating this thread after more than 3 years, but this is still one of the first threads to come up when googling, "covariate balance teffects stata". To anyone looking to investigate covariate balance after running -teffects-, you can do so using the -tebalance- command introduced in Stata 14 (as requested by Clyde in this thread).

[Philip Jones]

Yes, it is a useful command.

And, in case people come here looking for weighted means/proportions after using IPTW (not matching) instead of just the standardized differences, they could see this thread - https://www.statalist.org/forums/for...r-teffects-ipw

[Courtney Balentine]

The solution for finding means/proportions after iptw is very elegant but I still can’t find an approach that works after teffects psmatch. Anyone have something that works?