Prediction Models and Stabilised Inverse-Probability of Treatment Weighting

[Damien YP Foo]

Hi Statalist,

I am trying to build a prediction model to estimate the true burden of respiratory syncytial virus (RSV), the leading cause of hospitalisation in young children, in Western Australia (WA).

I have access to linked WA hospitalisation data, including perinatal, emergency department, vaccination, mortality, and laboratory pathology datasets. For each hospitalisation, we define a laboratory-confirmed RSV-associated admission as a hospital admission occurring within 4 days of the specimen date for a positive RSV test.

A challenge is that RSV testing is relatively uncommon. Approximately 10% of the hospitalised children are tested for RSV, and among those tested, around 20% are RSV-positive.

Variables:

• Predictors of interest include:
  • Maternal characteristics ($mat_vars): E.g., age, smoking during pregnancy, plurality, etc.
  • Child characteristics ($child_vars): E.g., sex, gestational age at birth, birthweight, etc.
  • Clinical outcomes ($clin_vars): E.g., bronchiolitis, pneumonia, etc.
  • Time-varying predictors ($time_vars): E.g., month/season/year of birth, month/season/year of admission, etc.
  • Geographical predictors ($geo_vars): E.g., socioeconomic status, remoteness
• Additional variables:
  • rsvtest (yes/no): Whether the child was tested for RSV
  • rsvpos (yes/no): RSV positivity among tested children

Proposed general steps of prediction model (in Stata):

1. Fit an elastic-net logistic model among RSV-tested children to select predictors. I am planning to use multiple different alphas, however, for simplicity, I will use an alpha of 1 (i.e., LASSO).

   • elasticnet logit rsvpos $mat_vars $child_vars $clin_vars $time_vars $geo_vars if rsvtest==1, rseed(123) alphas(1) selection(cv) cluster(child_ID) nolog

2. Document the variables selected by elastic-net logistic model:

   • global EN_vars_a1 "EN_vars"

3. Use the selected variables in a model for the predicted probability of RSV testing and derive stabilised inverse probability of testing weights (sIPTW). Note: The motivation is to account for differences between tested and untested children when estimating RSV positivity.

   • logistic rsvtest $EN_vars_a1, vce(cluster child_ID)

   • predict rsvtest_a1

   • quietly sum rsvtest

   • local rsvtest_mean=r(mean)

   • replace rsvtest_sIPTW_a1=(1-`rsvtest_mean')/(1-rsvtest_p_a1) if rsvtest==0

   • replace rsvtest_sIPTW_a1=`rsvtest_mean'/rsvtest_p_a1 if rsvtest==1

4. Fit a weighted logistic regression model for RSV positivity among tested children.

   • logistic rsvpos $EN_vars_a1 if rsvtest==1 [pw=rsvtest_sIPTW_a1], vce(cluster b_ID)

   • predict rsvpos_p_a1

5. Evaluate model performance (e.g., sensitivity, specificity, Youden index) and choose a probability cutoff. For simplicity, I will use a cutoff of 0.5.

   • estat classification, cutoff(.5)

6. Apply the model to untested children and classify probable RSV-associated hospitalisations.

   • gen predictedRSV=0

   • replace predictedRSV=1 if rsvpos_p_a1>=.5

7. Define the estimated "true" RSV burden as:

   • gen trueRSV=0

   • replace trueRSV=1 if rsvpos==1 | rsvtest==0 & predictedRSV==1

We also have plans to use the "trueRSV" variable for subsequent analyses.

Questions:

1. Does this seem like a reasonable framework for estimating the true burden of RSV in this setting?
2. Is the use of stabilised inverse probability of testing weights appropriate here, given that RSV status is only observed among tested children?
3. Are there alternative approaches (e.g., multiple imputation, two-stage selection models, semi-supervised learning, or other methods for outcome verification bias) that would be more appropriate?
4. Would it be preferable to estimate the RSV burden using the predicted probabilities directly (i.e., summing probabilities across untested children) rather than imposing a classification threshold such as 0.5?

Kind regards,
Damien

[Felix Bittmann]

I am not an expert with prediction and this is something I rarely do, so my advice is limited. I just want to point you to this rather elaborate package that might be worth a look. It could make things faster and simpler, as elasticnet is already implemented as a method: https://www.stata.com/meeting/us21/s...21_Cerulli.pdf