IPWRA (TEFFECTS Propensity Score): choosing a model building approach

[Michael McCulloch]

Hello, I'm seeking advice on optimal variable selection strategies for building Treatment and Outcomes models with IPWRA. I've obtained the 4 published papers identifiable on PubMed as having used the Stata command - TEFFECTS IPWRA -.

In two of these papers, the authors used a parsimonious model:
* Running IPWRA twice: first with all potential confounders, then a final IPWRA model, choosing all variables significantly associated with either Treatment or Outcome. (Moniodis & Townsend, 2017).
* Running IPWRA once, choosing for Treatment model those potential confounders significantly associated in logistic regression with receipt of Treatment. And, choosing for Outcome model, those associated with having the (dichotomous) outcome, again using logistic regression. (Anothaisintawee & Udomsubpayakul, 2016)

A third paper used ALL variables potentially associated with Treatment and Outcome, for both Treatment and Outcome models. (Criski & Culkin, 2015)

And, the fourth didn't specify their model or justify the choice. (Traxer & Wendt-Nordahl, 2015)

I'd appreciate any guidance as to navigate this choice in model building.
Michael


For those interested, the citations are below.
Comparison of extracorporeal photopheresis and alemtuzumab for the treatment of chronic lung allograft dysfunction.Moniodis A, Townsend K, Rabin A, Aloum O, Stempel J, Burkett P, Camp P, Divo M, El-Chemaly S, Mallidi H, Rosas I, Fuhlbrigge A, Koo S, Goldberg HJ.J Heart Lung Transplant. 2017 Mar 24. pii: S1053-2498(17)31732-1. doi: 10.1016/j.healun.2017.03.017. PMID: 28431983

Effect of Lipophilic and Hydrophilic Statins on Breast Cancer Risk in Thai Women: A Cross-sectional Study.Anothaisintawee T, Udomsubpayakul U, McEvoy M, Lerdsitthichai P, Attia J, Thakkinstian A.J Cancer. 2016 Jun 6;7(9):1163-8. doi: 10.7150/jca.14941. PMID: 27326260.

Preoperative JJ stent placement in ureteric and renal stone treatment: results from the Clinical Research Office of Endourological Society (CROES) ureteroscopy (URS) Global Study.Assimos D, Crisci A, Culkin D, Xue W, Roelofs A, Duvdevani M, Desai M, de la Rosette J; CROES URS Global Study Group.BJU Int. 2016 Apr;117(4):648-54. doi: 10.1111/bju.13250. Epub 2015 Sep 6.PMID: 26237735

Differences in renal stone treatment and outcomes for patients treated either with or without the support of a ureteral access sheath: The Clinical Research Office of the Endourological Society Ureteroscopy Global Study.Traxer O, Wendt-Nordahl G, Sodha H, Rassweiler J, Meretyk S, Tefekli A, Coz F, de la Rosette JJ.World J Urol. 2015 Dec;33(12):2137-44. doi: 10.1007/s00345-015-1582-8. Epub 2015 May 14.PMID: 25971204

[Matthew J. Baker]

Michael McCulloch --

You might consult the growing literature that combines model selection, inference about parameters, and inference about treatment effects. One recent example is Farrell (2015). Another useful example - that is perhaps more accessible - is Belloni, Chernozhukov, and Hansen (2014) - which is also about introducing model selection into estimation of treatment effects. The idea is to apply (grouped) lasso-type estimation methods in choosing explanatory variables scientifically that go in both the treatment and outcome models, and using results to calculate treatment effects.

In one of my projects, I had to cook up routines for running a group lasso with a (multinomial) logit model. Here is the github site. If you need to do something similar, I'd be happy to help!

Also, I had an exchange on this list about the inner workings of the teffects command, largely because I wanted to know how it worked so I could use it with lasso-type model selection methods. This exchange is here. It helped me quite a bit with learning how to do things out "by hand" if need be.

Just a few thoughts!

Matthew J. Baker

[Michael McCulloch]

Thank you Matthew. I'm looking now at the -ElasticRegress - command (specifically the -LASSOregress - subcommand). Looking at the underlying paper (Friedman, Hastie and Tibshirani, 2008), my theoretical statistical knowledge is not sufficient to follow the derivations, but it's wonderful to discover that this is a quantified approach to Directed Acyclic Graphs! DAGs were one of my favorite discoveries during dissertation work.

May I ask, in the following example from -LASSOregress-, is it correct that the results identify the variable <weight> to be a negative predictor of <mpg>, thus suitable for implementation in - TEFFECTS ipwra - , whereas <foreign> is not?

sysuse auto, clear
lassoregress mpg weight foreign

LASSO regression        Number of observations    =    74
        R-squared    =    0.6453
        alpha    =    1.0000
        lambda    =    0.4542
        Cross-validation MSE    =    12.4665
        Number of folds    =    10
        Number of lambda tested    =    100
                
mpg    Coef.
                
weight    .0054203
foreign    0
_cons    37.66363
                

And would adding more variables perform in the same manner, i.e. here that both <weight> and <length> are quantitatively identified as having predictive power suitable for - TEFFECTS ipwra - , whereas <headroom> and <foreign> are ruled out?

lassoregress mpg weight foreign headroom length

LASSO regression                       Number of observations     =         74
                                       R-squared                  =     0.6550
                                       alpha                      =     1.0000
                                       lambda                     =     0.4542
                                       Cross-validation MSE       =    12.2300
                                       Number of folds            =         10
                                       Number of lambda tested    =        100
------------------------------------------------------------------------------
         mpg |      Coef.
-------------+----------------------------------------------------------------
      weight |  -.0035564
     foreign |          0
    headroom |          0
      length |  -.0687995
       _cons |   44.96533
------------------------------------------------------------------------------

[Scott Howell]

Is there a Stata package like daggity or dagR to produce DAGs? I use the daggity web version and Tetrad but it would be cool to take care of everything in Stata.

[Michael McCulloch]

Hi Scott, yes, there is one, called DAG! I found it just now by entering into the command line "search acyclic, all".
Haven't used it, but would be interested to hear your feedback.