Computing standardized differences in Inverse probability of treatment weighting (IPTW)

[Evo Kurtz]

Im computing standardized differences (SD) to check balancing after i computed IPTW and stabilizing weights (SW) following http://onlinelibrary.wiley.com/doi/1.../sim.6607/full . When i compute SD for weighted case, for continuous variable according to given formula, i get SD in case of IPTW and stabilizing weights very similar (even though weights dont seem to be so similar) and for the case of dummies paper doesnt explicitly shows what is the formula (i wasnt successful in finding it in other literature) and by my calculation i get identical values for IPTW and SW. I used following code to compute them (both for continuous and dummy case). Am i making some computational error?

local continuous var1 var2 var3
tempvar w2 cm sddift sddifc
local w SWatt
//local w IPTWatt
foreach c of local continuous {

    qui gen `w2' = `w'*`w'    
    qui gen `cm' = (`c'*`w')
        
    qui sum `w' if treatment==1
    scalar sumwt`w' = r(sum)    
    qui sum `w2' if treatment==1
    scalar w2t = r(sum)
    qui sum `cm' if treatment==1
    scalar ctm`w' = r(sum)/sumwt`w'
    qui gen `sddift' = (`c' - ctm`w')*(`c' - ctm`w')*`w'    
    qui sum `sddift' if treatment==1
    scalar sddift`w' = r(sum)
    scalar sdt2`w' = (sumwt`w'/(sumwt`w'*sumwt`w' - w2t))*sddift`w'    
    
    qui sum `w' if treatment==0
    scalar sumwc`w' = r(sum)    
    qui sum `w2' if treatment==0
    scalar w2c`w' = r(sum)
    qui sum `cm' if treatment==0
    scalar ccm`w' = r(sum)/sumwc`w'
    qui gen `sddifc' = (`c' - ccm`w')*(`c' - ccm`w')*`w'
    qui sum `sddifc' if treatment==0
    scalar sddifc`w' = r(sum)
    scalar sdc2`w' = (sumwc`w'/(sumwc`w'*sumwc`w' - w2c))*sddifc`w'    
    
    scalar `c'_`w' = ((ctm`w' - ccm`w')/(sqrt((sdt2`w' + sdc2`w')/2)))*100
    scalar abs_`c'_`w' =  abs(`c'_`w')    
    drop `w2' `cm' `sddift' `sddifc'    
}
///


local dummy d1 d2 d3
tempvar w2 dm
local w IPTWatt
// local w SWatt
foreach d of local dummy {

    qui gen `w2' = `w'*`w'
    qui gen `dm' = (`d'*`w')
    sum `dm'        
    qui sum `w' if treatment==1
    scalar sumwt`w' = r(sum)    
    qui sum `w2' if treatment==1
    scalar w2t`w' = r(sum)
    qui sum `dm' if treatment==1
    scalar dtm`w' = r(sum)/(sumwt`w'*r(N))    
    di     dtm`w'
    qui sum `w' if treatment==0
    scalar sumwc`w' = r(sum)    
    qui sum `w2' if treatment==0
    scalar w2c`w' = r(sum)
    qui sum `dm' if treatment==0
    scalar dcm`w' = r(sum)/(sumwc`w' * r(N))
        
    scalar `d'_`w' = ((dtm`w' - dcm`w')/(sqrt((dtm`w'*(1-dtm`w')+dcm`w'*(1-dcm`w'))/2)))*100
    scalar abs_`d'_`w' =  abs(`d'_`w')
    drop `w2' `dm'    
}

[Philip Jones]

I have the same question. I hope that Stata 15 might add the calculation of standardized differences in the unweighted and weighted sample to its -teffects- commands. Automating this diagnostic step would be very helpful.

Evo Kurtz Have you moved forward with this? i.e., did you solve the problem?

Thanks, Phil

[Joerg Luedicke (StataCorp)]

Is tebalance summarize not doing what you are looking for? For example:

* Example data:
webuse cattaneo2, clear

* IPW model:
teffects ipw (bweight) ///
             (mbsmoke mmarried c.mage##c.mage fbaby medu, probit)

* Standardized mean differences and variance ratios:
tebalance summarize

tebalance was introduced in Stata 14 and supports other methods for balance checking as well (beyond standardized mean differences); see help tebalance for further information.

[Joseph Coveney]

tebalance was introduced in Stata 14 and supports other methods for balance checking as well (beyond standardized mean differences); see help tebalance for further information.

Thanks for pointing that out. I hadn't been aware of the command.

[Philip Jones]

Joerg Luedicke (StataCorp) Oops! I wasn't aware of it either. Thank you very much for pointing that out!

I am still curious whether this solved the original question. I presume it should have...

[Ben Lazrus]

Dear Statalist,

As far as I can tell teffects ipw doesn't accept multilevel models to calculate the inverse probability of treatment weights, so this has to be done manually
I'm trying to calculate the standardized differences for multinomial categorical variable in the weighted sample.
This paper provides useful information on how to calculate this in the unweighted sample, but I'm not sure how to adapt this to my weighted sample.
https://support.sas.com/resources/pa...2/335-2012.pdf
Any suggestions?

Example code for the unweighted sample:

webuse nhanes2, clear
recode sex (1 = 0 "Male") (2 = 1 "Female"), gen(sex_recode)
drop if hlthstat == .
melogit sex_recode        i.race i.hlthstat ||sizplace:
predict double ps 
generate double ipw = (sex_recode/ps) + (1-sex_recode)/(1-ps)
stddiff i.race i.hlthstat, by(sex_recode)

But unsure how to calculate this in the weighted sample. Any help would be most appreciated. Philip Jones Joerg Luedicke (StataCorp)

Kind regards,
Ben