Changing outcome model in: teffects psmatch

[Francis Clark]

Hello all,

According to the [TE] teffects psmatch - Propensity-score matching help-file:

"Syntax:

teffects psmatch (ovar) (tvar tmvarlist [, tmodel]) [if] [in] [weight] [, stat options]

ovar is a binary, count, continuous, fractional, or nonnegative outcome of interest."

Furthermore, the help-file indicates: "teffects psmatch accepts a continuous, binary, count, fractional, or nonnegative outcome."

You can even access teffects psmatch by clicking:

Statistics -> Treatment Effects -> Outcome type (continuous, binary, count, etc.) -> Propensity score matching.

However, once you enter this menu... there are no options to change the outcome model. I don't want to change the treatment model (where there are indeed options to change it) - but I want to change the outcome model.

For example, if you click "IPW-RA" - you can select various outcome models from a menu. Same with "augmented IPW." I don't want to use IPW-RA or augmented IPW, but I am just illustrating what is available through these menus versus what is (un)available under propensity score matching. I am confused by this because there is both an option for different outcome types (using propensity score matching) under "Statistics -> Treatment Effects" and the help-file itself also discusses different types of 'ovar' (outcome variables). If I try to add different outcome variable types into my code directly:

. teffects psmatch (outcome, poisson) (exposure confounders), atet
options not allowed
The outcome model is misspecified.
r(101);

. teffects psmatch (outcome, logit) (exposure confounders), atet
options not allowed
The outcome model is misspecified.
r(101);

I am assuming when I just run: teffects psmatch (outcome) (exposure confounders), atet - with a 0,1 binary outcome variable, it is automatically assuming the use of a logit model (as opposed to a linear model)? Therefore, I find it bizarre that when I specify "logit" - it says the outcome model is misspecified. Ideally, I want to use a Poisson model for count data with propensity score matching... but it doesn't appear that it allows you to do so.

How do you change the outcome model using teffects psmatch?

Thanks for any insights!

[Francis Clark]

Does anyone have any insights? Thanks so much!

[Ariel Karlinsky]

Your syntax is incorrect and you may have some fundamental misunderstanding of the command and the estimation.
The outcome variable can be anything, but it's distrbution is of no consuqnace for propensity score matching - as matching is essentially a non-paramteric estimation method. In propensity score matching, the only paramteric estimation is of the propensity score - which can be either logit or probit. to switch between the two, the syntax is:

teffects psmatch (outcome) (exposure confounders, logit)
teffects psmatch (outcome) (exposure confounders, probit)

In short, for psmatch - the first brackets will only contain the outcome variable, the second brackets will contain the exposure variable, it's determinants and other model options.

[Francis Clark]

Your syntax is incorrect and you may have some fundamental misunderstanding of the command and the estimation.
The outcome variable can be anything, but it's distrbution is of no consuqnace for propensity score matching - as matching is essentially a non-paramteric estimation method. In propensity score matching, the only paramteric estimation is of the propensity score - which can be either logit or probit. to switch between the two, the syntax is:

teffects psmatch (outcome) (exposure confounders, logit)
teffects psmatch (outcome) (exposure confounders, probit)

In short, for psmatch - the first brackets will only contain the outcome variable, the second brackets will contain the exposure variable, it's determinants and other model options.

Hello Ariel,

Thank you for your response!

My syntax is exactly the same as yours (although I added something to the first bracket to demonstrate a potential limitation in the teffects psmatch package.)

As mentioned, I did not want to change the exposure model (for the propensity score) - I wanted to use the default (logit for a binary exposure variable). Specifically, I did not specify "logit" in the second bracket because I wanted to use the default exposure model. I also do not understand how the "outcome" model is not important - as after matching, you still need to fit a model (on the matched sets) in order to estimate an effect measure of interest. For example, if I wanted to do matching outside of teffects, I could match people manually (after estimating the propensity scores, manually) - and fit any type of outcome model that I want (e.g., Poisson, logit, linear, probit) to estimate different effect measures (while taking into account the fact that matching took place).

My point is: I do not want to estimate an odds ratio (OR) using a logistic outcome model - which is what this command appears to be doing by default. I would rather use Poisson regression for the outcome model. In the attached screenshot from the Stata manuals - you can see that teffects ipwra (for example) allows you to set both an outcome and exposure (treatment) model in either bracket. Therefore, I would think that I could also change the outcome model in psmatch as well (in fact, the documentation actually says that you can/the drop-down Stata menus also seem to indicate that you can). But by replicating the "structure" of the command from IPWRA or AIPW (augmented IPW)... it tells you that you cannot change the outcome model in teffects psmatch. My question is: why not?

[Ariel Karlinsky]

I am less familiar with ipwra and ra - What I said previously still holds. The outcome is non-paramteric, it is merely the average of the outcome of each observation minus the outcome matched with other observations - either by "neighbors" or by propensity score.
I think a good way to understand it is by picturing an imaginary data with several variables: outcome, treatment, and treatment determining variables. Let's assume you were given the option, after estimating the propensity score, to estimate the outcome as a poisson. what does that mean, and how will that differ from other outcome models?

[Francis Clark]

Hello Ariel,

I appreciate you taking the time to reply.

I'm still slightly confused then - what does the coefficient (in the final model) represent then... in the model output of teffects psmatch?

In the Stata manuals - the examples they provide are with a continuous outcome... so the final coefficient is the difference in the potential outcome means between the matched exposed vs. the matched unexposed. However, my outcome is binary, and it appears to be giving a coefficient which I then exponentiate to get what I (assume/think) is an odds ratio. But perhaps I am not understanding this as well as I think I am! Hmm.

[Joerg Luedicke (StataCorp)]

Francis, please have a look at this thread where I provide an interpretation of the average treatment effect in the context of teffects psmatch with a binary outcome variable.

Joerg

[Francis Clark]

Hello Joerg,

Thanks so much for responding! This is interesting, surprising (to me), and useful.

As you mentioned in the other post (regarding interpreting the coefficient from a teffects psmatch output with a binary outcome variable):

The average treatment effect from the propensity score matching estimator is the average of the differences between observed and potential outcomes. If the outcome variable is binary, then this average can be interpreted as a difference in probability. In this example, the result of ATE=0.043 indicates that smokers have a higher probability of giving birth to a low birth weight baby than non-smokers, and the average difference in probability is 0.043. Expressing this result on a percentage scale, we could say that the chance of giving birth to a low birth weight baby is higher by 4.3 percentage points for smokers, compared to non-smokers (notice that this is a difference in percentage points, not a relative difference in percent).

First, I am assuming the same interpretation applies for the ATET - as described above for the ATE (with the exception that we are only using observed and potential outcomes for those who were in fact treated)?

Second, does your description correspond to what is typically described as a "risk difference" (RD) in epidemiology? In which case, a coefficient of 0 would be null ("no effect"), anything < 0 would be a "protective" effect of the treatment (i.e., the risk/probability of experiencing the outcome in the treated is lower than the risk/probability of experiencing the outcome in the untreated), and anything > 0 would be indicative of an increased "risk" given treatment. This is assuming a binary treatment as well (where treatment = 1 if treated and treatment = 0 if untreated). I think the below formulae from the teteffects intro advanced documentation are equally applicable to both continuous and binary outcomes then?

Third, Is there any documentation (Stata or otherwise) and/or published examples out there that you could refer me to - to further develop my understanding with respect to the interpretation of the model output from teffects psmatch with a binary outcome variable?

Thanks so much!

[Joerg Luedicke (StataCorp)]

First, I am assuming the same interpretation applies for the ATET - as described above for the ATE (with the exception that we are only using observed and potential outcomes for those who were in fact treated)?

Yes, that is correct.

Second, does your description correspond to what is typically described as a "risk difference" (RD) in epidemiology?

Yes, I guess you could call it a "risk difference" where risk here would be defined as the averaged probability of the outcome variable being 1 (for a 0/1 coded outcome variable).

Third, Is there any documentation (Stata or otherwise) and/or published examples out there that you could refer me to - to further develop my understanding with respect to the interpretation of the model output from teffects psmatch with a binary outcome variable?

Off the top of my head, I am not aware of a reference discussing binary outcome variables in particular, but there is probably something out there somewhere. However, there really is nothing special about having a binary vs. a continuous outcome variable in this context. All there is to understand is that there is no model for the outcome process, and the average treatment effect is simply the average of the differences between observed and potential outcomes. For a 0/1 coded outcome variable, these differences are bounded by -1 and 1 and then the average of theses differences is an estimate of the ATE (or ATET if restricted to the treated) that can be interpreted as an average difference in probability of the outcome being 1.

I hope this helps,
Joerg

[Francis Clark]

Very helpful and clear Joerg! I understand completely now. Thanks to both you and Ariel for these insights. I think that they will also be useful to future individuals who use teffects psmatch with a binary outcome.