Propensity Score Matching for Multiple treatments - teffects psmatch doesn't work with mlogit?

[Rythia Afkar]

Hello,

I am trying to run propensity score for multiple treatments (with 4 treatments). The manual for teffects psmatch stated that this command also works for multivariate treatments, where multinomial logit is used. However when I try to use this command as below (note: my outcome variable is pcexp, my treatment variable is treatment, and my covariates are head*, good*, and listrik):

teffects psmatch (pcexp) (treatment head* good* listrik, logit)

I get the error message below:

treatment variable treatment must have 2 levels, but 4 were found

Can anyone suggest anything or any other hints are appreciated.

Best regards,
Rythia

[Jeff Wooldridge]

Did you try using "mlogit" in place of "logit"?

[Rythia Afkar]

Yes I did, it gave me another error message:
option mlogit not allowed

[Rythia Afkar]

these are the complete error messages:

option mlogit not allowed
The treatment model is misspecified.
r(198);

[Jeff Wooldridge]

Looks like mlogit is allowed only with the ipw option. That's a bit puzzling. It should work with psmatch and ipwra, too.

[Rythia Afkar]

I thought so too. I want to do analysis on multiple overlapping treatment effects like in Bradley & Migali (2012) "The joint evaluation of multiple educational policies: the case of specialist schools and excellence in Cities policies in Britain", but my analysis is on other sector (not on education).

In my multiple-treatment analysis I have a categorical variable 'treatment':
0 for not receiving any program
1 for receiving program 1 only
2 for receiving program 2 only
3 for receiving program 1 and 2

I thought that I can apply teffects psmatch command to get the propensity score for case multiple treatment like this, as the manual said it accommodates multivariate treatment using multinomial logit. But so far I always get the error as I described above.

[Melissa Garrido]

Page 98 of the manual states that you shouldn't be able to use teffects psmatch for multi-valued treatments: "You can use teffects ra, teffects ipw, teffects ipwra, and teffects aipw to estimate
multivalued treatment effects. However, the theory developed in Abadie and Imbens (2006, 2012) has not been extended to handle multivalued treatments, so you cannot use teffects nnmatch or
teffects psmatch in these cases."

Coming up with matches across multiple treatment groups is not impossible, but it can be difficult, so weighting might be a better option for you to pursue anyway.

[Rythia Afkar]

Ah ok, thanks Melissa, I will explore teffects ra, teffects ipw, teffects ipwra, and teffects aipw.
Also thank you Jeff !

[Jeff Wooldridge]

Thanks Melissa. In that case, I would recommend combining regression adjustment and ipw.

[Rythia Afkar]

Thanks again Melissa and Jeff for your replies. I tried with teffects ra and teffects ipw. They worked perfectly. Below is the snapshot of the output:
Iteration 0: EE criterion = 9.296e-18
Iteration 1: EE criterion = 1.000e-31

Treatment-effects estimation Number of obs = 11626
Estimator : inverse-probability weights
Outcome model : weighted mean
Treatment model: (multinomial) logit
------------------------------------------------------------------------------
| Robust
difftotcons | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ATET |
treatment |
(1 vs 0) | .2754387 .1574454 1.75 0.080 -.0331487 .5840261
(2 vs 0) | .1443831 .096309 1.50 0.134 -.0443791 .3331453
(3 vs 0) | .2632009 .1020774 2.58 0.010 .0631329 .4632689
-------------+----------------------------------------------------------------
POmean |
treatment |
0 | .4103619 .0937795 4.38 0.000 .2265574 .5941664
------------------------------------------------------------------------------

From the output above, I can get the atet for (1 vs 0), (2 vs 0), and (3 vs 0).
Now I have a question of how do I compare treatment: (1 vs 2), (1 vs 3), and (2 vs 3) like in the paper of Bradley & Migali (2012) "The joint evaluation of multiple educational policies: the case of specialist schools and excellence in Cities policies in Britain".
As far as I understood, (maybe) we can do these steps:

1. Run multinomial logit to estimate propensity score for each treatment.
2. Run psmatch2 only for treatment 1 and 2 to get the atet (1 vs 2).
3. Run psmatch2 only for treatment 1 and 3 to get the atet (1 vs 3).
4. Run psmatch2 only for treatment 2 and 3 to get the atet (2 vs 3).

Any advice or insight would be highly appreciated.

Thanks,
Rythia

[Melissa Garrido]

Within teffects ra and teffects ipw, you should be able to specify the comparison category for your ATETs. There is an option control(#) where you can specify that category 2 or 3 should be the comparison category. There's a nice example on p. 104 of the treatment effects manual.

Hope this helps!

[Rythia Afkar]

thanks so much Melissa, this information helps a lot!

[Rythia Afkar]

Hello,

I think I confused myself with multivalued vs multiple treatments. As I understood from the manual, teffects ra / teffects ipw / teffects ipwra / teffects aipw can be used to estimate multivalued treatment case. But I think in my case, I have both multivalued and multiple treatments. I have 4 groups:

group 0 for not receiving any program
group 1 for receiving program 1 only
group 2 for receiving program 2 only
group 3 for receiving program 1 and 2

So for group 3, they have multiple treaments (receiving program 1 and 2) .

Can I use teffects ra / teffects ipw / teffects ipwra / teffects aipw for my case above even though the group 3 has multiple treatments?
(I do not find example or discussion about multiple treatments in the manual).

Thanks in advance for your help.

Best regards,
Rythia

[Melissa Garrido]

Hi Rythia,
Yes, you can still run the same models. You can think of the combination of programs 1 and 2 as a single unique treatment. You'll just need to make sure you are thinking of your 4 treatment groups as multinomial (unordered) versus ordinal.
Hope this helps!
Melissa

[Rythia Afkar]

Hi Melissa,
Thanks so much for your reply! Yes, that helps a lot.

In the case of panel data, I just wanna confirm that I can combine the models using DID (diff in diff), is that right?
So for example I use below command: teffects ra (difftotcons troof tfloor septic twal toilet water lighting fcook hhsize sltblt askeskin pregnant y*t*r) (treatment), atet

with:
- selected outcome: difftotcons = consumption_year_t2 - consumption_year_t1 (difference in total consumption year 2 and year 1)
- covariates: troof tfloor septic twal toilet water lighting fcook hhsize sltblt askeskin pregnant y*t*r
- treatment: 4 groups of treatment (as explained above)

Secondly, I wonder if you know of any papers which use regression adjustment / IPW method for multivalued and multiple treatments (as in my case)? I have searched via google scholar, but do not yet find any, I found some papers using GPS (Generalized Propensity Score) though.

Thanks again in advance for your help!

Best regards,
Rythia