Running a univariate logistic regression after psmatch 2

[Zach Zottola]

Hello Statalist,

Let me preface with the fact that I am new to statistical coding, but as the heading shows I am having difficulty running a proper univariate logistic regression on a 1:1 matched sample using psmatch2 on Stata 17. I understand this may be slightly unorthodox but the main goal is to obtain an odds ratio for my outcome of interest. That being said my current match code reads as such:

psmatch2 treatment covariate1 covariate#, out(outcome variable) caliper(0.01) noreplacement descending logit ate index

After the match I attempted to run the regression using the code:

logistic outcome treatment if _support==1

as I thought that simply using the observations that are on the common support of the match would give the match sample and suffice. However, I have found in my searches (https://www.statalist.org/forums/for...opensity-score) that the following codes:

outcome treatment [pw =_weight]

or

logistic outcome  treatment if _weight==1

may be the more proper way to run this code. So, now I am a bit confused on what exactly the variable_support and _weight are giving and why is running the logistic regression on weight the proper way to do so?

Thanks!

[Fei Wang]

Zach, the code below is correct. In 1-1 matching, a subject may be used multiple times and only weighted regressions can reveal the true matching process. Anyone who fails to match others will be assigned missing weights and be dropped from the logit regression.

logistic outcome treatment [pw=_weight]

Add: just realized you specified -noreplacement- and -descending- options. Then each subject would at most be used once for matching and "_weight" can only be 1 or missing. In this particular case, the following codes should give you identical results to the above one. But the code above is correct in general cases.

logistic outcome treatment if _weight == 1
logistic outcome treatment if _support == 1

[Zach Zottola]

Fei, Thank you for the help! As an update I did run both of the codes and when running the logistic regression by _weight ==1 and by _support ==1 I do get different results. Notably the match code above gives two groups of 3,251 observations on the common support so when I run the regression by _support the outcome shows that 6,502 observations were analyzed, but when running based on _weight only 3,729 observations are analyzed.

I looked further into how many observations have a weight ==1 compared to the observations on common support and got the result below. It looks like only 478 treated observations were used as indicated by the weight

 psmatch2: |   psmatch2: Common
 Treatment |        support
assignment | Off suppo  On suppor |     Total
-----------+----------------------+----------
 Untreated |         0      3,251 |     3,251 
   Treated |    31,074      3,251 |    34,325 
-----------+----------------------+----------
     Total |    31,074      6,502 |    37,576

. tab _treated _weight

           | psmatch2:
           | weight of
 psmatch2: |  matched
 Treatment |  controls
assignment |         1 |     Total
-----------+-----------+----------
 Untreated |     3,251 |     3,251 
   Treated |       478 |       478 
-----------+-----------+----------
     Total |     3,729 |     3,729

Do you know why these may be different even though I ran my match with no replacement?

[Fei Wang]

Zach, the definitions of "_weight" and "_support" turn out to be more subtle than I thought. I'm trying to extrapolate with the following example which essentially replicates your matching options. Line 1 shows that the first untreated obs (id 1) is matched to a treated obs with id 55 (you may find its information in line 55). It's on support and _weight = 1. In line 2, id 2 fails to match anyone from the treatment group, as its nearest neighbor is id 56 (55 has been occupied) and the pscore difference is far beyond the specified caliper. Then it's off support and _weight is missing. The first subtle case appears in line 5 where id 5 is matched with id 59, the pscore difference is within the caliper and it's on support, BUT _weight is missing. My guess is: Line 59 shows that id 59 is matched to id 6, and ids 5 and 6 are competing id 59 in a 1-1 matching. As pscore diff is smaller for the 59-6 match, this pair is retained and the other is assigned a missing weight. Similar cases are marked in red and share this common feature. But that's only my extrapolation, because I found contradictions. Lines 31 and 73 compete id 31 but both weights are retained; lines 31 and 74 compete id 74 but both weights are retained. Surprisingly I'm not able to find any formal documentation of such a widely-used command, and cannot figure out if it means something else or simply a bug. Given limited number of choices (the Stata official command -teffects psmatch- troubles me a lot with error messages related to -osample()-), I would still use -psmatch2- and conduct consequent analysis with -[pw=_weight]- which seems to be a conservative choice.

. sysuse auto, clear
(1978 Automobile Data)

. quietly psmatch2 foreign price mpg, caliper(0.01) noreplacement descending logit ate

. sort _id

. list _id _n1 _weight _treat _support _pscore _pdif

     +-----------------------------------------------------------------------+
     | _id   _n1   _weight    _treated      _support     _pscore       _pdif |
     |-----------------------------------------------------------------------|
  1. |   1    55         1   Untreated    On support   .81663669   .00776549 |
  2. |   2     .         .   Untreated   Off support   .81448508           . |
  3. |   3    58         1   Untreated    On support   .60510102   .00069901 |
  4. |   4     .         .   Untreated   Off support   .53917046           . |
  5. |   5    59         .   Untreated    On support   .53381466   .00980947 |
     |-----------------------------------------------------------------------|
  6. |   6     .         .   Untreated   Off support   .51670193           . |
  7. |   7    60         1   Untreated    On support    .5026768   .00232283 |
  8. |   8    62         1   Untreated    On support   .40520764   .00320632 |
  9. |   9     .         .   Untreated   Off support   .40302399           . |
 10. |  10    64         1   Untreated    On support   .37352019   .00352822 |
     |-----------------------------------------------------------------------|
 11. |  11    68         1   Untreated    On support   .35207438   .00066937 |
 12. |  12     .         .   Untreated   Off support   .31169477           . |
 13. |  13    70         1   Untreated    On support   .28493424   .00026151 |
 14. |  14     .         .   Untreated   Off support   .28368927           . |
 15. |  15    71         1   Untreated    On support   .25719977   .00655742 |
     |-----------------------------------------------------------------------|
 16. |  16     .         .   Untreated   Off support   .24122494           . |
 17. |  17     .         .   Untreated   Off support   .24092777           . |
 18. |  18     .         .   Untreated   Off support   .24087912           . |
 19. |  19     .         .   Untreated   Off support   .22691245           . |
 20. |  20     .         .   Untreated   Off support   .21322627           . |
     |-----------------------------------------------------------------------|
 21. |  21     .         .   Untreated   Off support   .20655899           . |
 22. |  22     .         .   Untreated   Off support   .19571024           . |
 23. |  23     .         .   Untreated   Off support   .18465087           . |
 24. |  24     .         .   Untreated   Off support   .18345238           . |
 25. |  25    72         1   Untreated    On support   .16923841   .00052232 |
     |-----------------------------------------------------------------------|
 26. |  26     .         .   Untreated   Off support   .16468834           . |
 27. |  27     .         .   Untreated   Off support   .15574173           . |
 28. |  28     .         .   Untreated   Off support   .14385061           . |
 29. |  29    73         .   Untreated    On support   .13990358   .00634236 |
 30. |  30     .         .   Untreated   Off support   .13831072           . |
     |-----------------------------------------------------------------------|
 31. |  31    74         1   Untreated    On support   .13048249   .00236589 |
 32. |  32     .         .   Untreated   Off support   .13018097           . |
 33. |  33     .         .   Untreated   Off support   .12497112           . |
 34. |  34     .         .   Untreated   Off support   .12468051           . |
 35. |  35     .         .   Untreated   Off support   .12069974           . |
     |-----------------------------------------------------------------------|
 36. |  36     .         .   Untreated   Off support   .11625587           . |
 37. |  37     .         .   Untreated   Off support   .11320104           . |
 38. |  38     .         .   Untreated   Off support   .10977742           . |
 39. |  39     .         .   Untreated   Off support   .10949224           . |
 40. |  40     .         .   Untreated   Off support   .10723004           . |
     |-----------------------------------------------------------------------|
 41. |  41     .         .   Untreated   Off support   .10654394           . |
 42. |  42     .         .   Untreated   Off support   .10543485           . |
 43. |  43     .         .   Untreated   Off support   .10403845           . |
 44. |  44     .         .   Untreated   Off support   .09641263           . |
 45. |  45     .         .   Untreated   Off support   .08778698           . |
     |-----------------------------------------------------------------------|
 46. |  46     .         .   Untreated   Off support   .08634989           . |
 47. |  47     .         .   Untreated   Off support   .08530629           . |
 48. |  48     .         .   Untreated   Off support   .08400691           . |
 49. |  49     .         .   Untreated   Off support   .08244444           . |
 50. |  50     .         .   Untreated   Off support   .07581779           . |
     |-----------------------------------------------------------------------|
 51. |  51     .         .   Untreated   Off support   .06310915           . |
 52. |  52     .         .   Untreated   Off support   .05004451           . |
 53. |  53     .         .     Treated   Off support   .96690884           . |
 54. |  54     .         .     Treated   Off support   .85281886           . |
 55. |  55     1         1     Treated    On support   .82440218   .00776549 |
     |-----------------------------------------------------------------------|
 56. |  56     .         .     Treated   Off support   .68728644           . |
 57. |  57     .         .     Treated   Off support   .64515746           . |
 58. |  58     3         1     Treated    On support   .60580004   .00069901 |
 59. |  59     6         1     Treated    On support   .52400519   .00730326 |
 60. |  60     7         1     Treated    On support   .50499963   .00232283 |
     |-----------------------------------------------------------------------|
 61. |  61     .         .     Treated   Off support   .43546533           . |
 62. |  62     8         1     Treated    On support   .40841396   .00320632 |
 63. |  63    10         .     Treated    On support   .38177621   .00825602 |
 64. |  64     .         .     Treated   Off support   .36999197           . |
 65. |  65     .         .     Treated   Off support   .36522908           . |
     |-----------------------------------------------------------------------|
 66. |  66    11         .     Treated    On support      .36006   .00798562 |
 67. |  67     .         .     Treated   Off support   .35541701           . |
 68. |  68     .         .     Treated   Off support   .35140501           . |
 69. |  69     .         .     Treated   Off support    .3351562           . |
 70. |  70    13         1     Treated    On support   .28519575   .00026151 |
     |-----------------------------------------------------------------------|
 71. |  71    15         1     Treated    On support   .25064235   .00655742 |
 72. |  72    25         1     Treated    On support    .1687161   .00052232 |
 73. |  73    31         1     Treated    On support   .13356122   .00307873 |
 74. |  74    32         1     Treated    On support    .1281166   .00206437 |
     +-----------------------------------------------------------------------+

[Zach Zottola]

Fei, this is incredibly helpful thank you!

[Fatimah Gh]

I want to do the propensity score for multivariate to match my data but I'm not sure if I can't do it with any covariate without minding the significant p values. And I want to know which covariates I should include.
*Not all of the covariates are similar some are continuous, and some are category with 2 or 3 values. Dose the difference in the values of variables affect the result or not?