Logistic regression: Effect modification

[Marvin Aliaga]

Hello guys,

I think I have a moderator variable in my logistic model. My DV is whether or not patients in a mental institution were involved in use of force (UOF) incidents with the staff. The ID variables are:

1. Score: An score we assigned to all patients that assess how violent they are (continuous)
2. PastAdm: Whether or not patients had a previous addition to the facility (cat).

I think my Score variable is moderating and changing the direction of the effect of PastAdm on UOF. I create a categorical version of my Score variable (low score- medium -high score), and then regress the PastAdm variable on the UOF for the groups (stratas). I see that low Score patients (not violent) who had a history of previous admissions are more likely to be involve in UOF compare to low score patients who do not have a history of admission. Contradictory, among high Score patients (violent), those with a previous Admission are less likely to be involved in UOF than those without a previous admission. In other words, high score patient who enter the system for the fist time (no previous admission) are more likely to be involved in UOF than those who were with us before. It seems that violent patients (high Score) who were admitted before already know the system and therefore do not want to be involved in UOF (for the consequences)In summary, the relation of PastAdm and UOF change based on ICS. The direction of the influence change: positive for low ICS and negative for high ICS.

1. How can I account for this in my model interaction/effect modification/or whatever it is?
2. Can I included an interaction effect term such as Score*PastAdm? Or Should I Run two separate models one for low and one for high score?
3. How can I represent this in a graph?

. * Only PastAdm as IV
. logistic UOF15 EverInc

Logistic regression                             Number of obs     =     47,760
                                                LR chi2(1)        =     147.85
                                                Prob > chi2       =     0.0000
Log likelihood = -8724.0089                     Pseudo R2         =     0.0084

------------------------------------------------------------------------------
       UOF15 | Odds Ratio   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     EverInc |   1.718847    .077605    12.00   0.000      1.57328    1.877883
       _cons |   .0352347   .0012392   -95.13   0.000     .0328878    .0377491
------------------------------------------------------------------------------

. * Only Score as IV
. logistic UOF15 i.ICSC

Logistic regression                             Number of obs     =     37,587
                                                LR chi2(2)        =    3061.50
                                                Prob > chi2       =     0.0000
Log likelihood = -6666.0196                     Pseudo R2         =     0.1868

------------------------------------------------------------------------------
       UOF15 | Odds Ratio   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        ICSC |
          2  |   2.934757   .2073414    15.24   0.000     2.555257     3.37062
          3  |   22.15072   1.423205    48.22   0.000     19.52978     25.1234
             |
       _cons |   .0161902   .0008944   -74.64   0.000     .0145288    .0180415
------------------------------------------------------------------------------

. * PastAdm and Score as ID - the effect of PastInc changed direction
. logistic UOF15 i.ICSC EverInc

Logistic regression                             Number of obs     =     37,587
                                                LR chi2(3)        =    3075.65
                                                Prob > chi2       =     0.0000
Log likelihood = -6658.9477                     Pseudo R2         =     0.1876

------------------------------------------------------------------------------
       UOF15 | Odds Ratio   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        ICSC |
          2  |   3.032999   .2159513    15.58   0.000     2.637948    3.487212
          3  |   23.58226    1.56941    47.49   0.000     20.69843    26.86787
             |
     EverInc |   .8266141   .0417467    -3.77   0.000     .7487112    .9126228
       _cons |   .0174652   .0010203   -69.28   0.000     .0155757     .019584
------------------------------------------------------------------------------

. * regression by strata
. bysort ICSC: logistic UOF15 EverInc if ICSC !=.

--------------------------------------------------------------------------------------------------------------------------------------------------
-> ICSC = 1

Logistic regression                             Number of obs     =     20,901
                                                LR chi2(1)        =       3.60
                                                Prob > chi2       =     0.0577
Log likelihood = -1706.9558                     Pseudo R2         =     0.0011

------------------------------------------------------------------------------
       UOF15 | Odds Ratio   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     EverInc |   1.234472    .136636     1.90   0.057     .9937267    1.533541
       _cons |   .0147404   .0011193   -55.54   0.000     .0127021    .0171057
------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------------------------------------------------------
-> ICSC = 2

Logistic regression                             Number of obs     =     11,905
                                                LR chi2(1)        =       1.98
                                                Prob > chi2       =     0.1598
Log likelihood = -2196.8736                     Pseudo R2         =     0.0004

------------------------------------------------------------------------------
       UOF15 | Odds Ratio   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     EverInc |   .8823108   .0783792    -1.41   0.159     .7413205    1.050116
       _cons |   .0510823    .003409   -44.57   0.000     .0448192    .0582205
------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------------------------------------------------------
-> ICSC = 3

Logistic regression                             Number of obs     =      4,781
                                                LR chi2(1)        =      30.60
                                                Prob > chi2       =     0.0000
Log likelihood = -2744.1011                     Pseudo R2         =     0.0055

------------------------------------------------------------------------------
       UOF15 | Odds Ratio   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     EverInc |   .6693567   .0480926    -5.59   0.000     .5814329    .7705763
       _cons |   .4783626   .0287598   -12.26   0.000     .4251888    .5381862
------------------------------------------------------------------------------

[Carlo Lazzaro]

Marvin:
I would test

...an interaction effect term such as Score*PastAdm...

[Clyde Schechter]

And, to support the later use of -margins- and -marginsplot- to facilitate interpretation of your findings, I would code the interaction as i.Score##i.PastAdm.

If this modeling works out, consider going back to using a continuous score variable--grouping into categories usually discards useful information.

[Marvin Aliaga]

Thank you so much for the comments and I am sorry for the late reply. A couple of notes: Score=ICS and EverInc=PastAdm. Also I had to delete my observations with negatives Scores values since the model did not accept negative values.

Below you can see the results for the model including the interaction term. First treating the Score variable as a categorical variable and the second model treating it as a continuous variable. However, the results are slightly different. My interpretation is that overall as ICS increase the probability of being involved in UOF increase. The PastAdmin variable in the first model is significant but in the second model is not. Finally, the interaction effect; I interpret this as: as Scores increase for patients who had a previous admission, the probability of being involved in UOF decrease compare to those who are first timers.

1. Why the two models have different results for the PastAdmin variable?
2. I used the margins and marginsplot commands before but never for interaction. Clyde Schechter Can you help me with this? How would be the command.

.  logistic UOF15 i.ICS##i.EverInc
note: 30.ICS#1.EverInc != 0 predicts success perfectly
      30.ICS#1.EverInc dropped and 10 obs not used


Logistic regression                             Number of obs     =     35,913
                                                LR chi2(58)       =    3727.86
                                                Prob > chi2       =     0.0000
Log likelihood = -6060.4652                     Pseudo R2         =     0.2352

------------------------------------------------------------------------------
       UOF15 | Odds Ratio   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         ICS |
          2  |   1.871998   .7750835     1.51   0.130      .831521    4.214417
          3  |   1.556714   .6324308     1.09   0.276     .7021012    3.451581
          4  |   2.241695   .8413562     2.15   0.031     1.074243    4.677898
          5  |   2.998647   1.101976     2.99   0.003     1.459205    6.162181
          6  |    3.20624   1.188489     3.14   0.002     1.550503     6.63009
          7  |   4.783018   1.714646     4.37   0.000     2.368962    9.657084
          8  |   4.689889    1.79355     4.04   0.000      2.21635    9.924002
          9  |   6.008549    2.29138     4.70   0.000     2.845517    12.68756
         10  |   12.09991   4.558186     6.62   0.000     5.782614    25.31864
         11  |   14.13634   5.547493     6.75   0.000     6.550911    30.50508
         12  |   26.65116   9.792374     8.93   0.000     12.97064    54.76097
         13  |   29.14046   10.97276     8.96   0.000     13.93089    60.95564
         14  |   46.21495   17.35544    10.21   0.000     22.13729     96.4807
         15  |   38.29239     15.089     9.25   0.000     17.68883    82.89451
         16  |   60.67193   23.95828    10.40   0.000       27.981    131.5565
         17  |   30.65967   13.11146     8.00   0.000     13.26036    70.88913
         18  |   51.89047   21.97888     9.32   0.000       22.623    119.0214
         19  |   83.42391   35.68332    10.34   0.000     36.07463    192.9209
         20  |   116.7536   48.35215    11.49   0.000     51.85071    262.8969
         21  |   64.08473   30.20275     8.83   0.000     25.44417    161.4064
         22  |   211.2683   92.76918    12.19   0.000     89.34428    499.5766
         23  |   123.2399    66.9585     8.86   0.000     42.48911    357.4578
         24  |   591.5496   296.2183    12.75   0.000     221.6931    1578.447
         25  |   274.6488   158.1657     9.75   0.000     88.83514    849.1232
         26  |    862.679   478.0672    12.20   0.000     291.1672    2555.972
         27  |   258.8037   183.8634     7.82   0.000     64.30527    1041.584
         28  |   554.5793   363.0391     9.65   0.000     153.7268    2000.681
         29  |    88.7327   71.27368     5.58   0.000      18.3808    428.3543
         30  |     739.39   846.8544     5.77   0.000     78.33514    6978.956
             |
   1.EverInc |   2.906886   1.422055     2.18   0.029     1.114342    7.582939
             |
 ICS#EverInc |
        2 1  |    .445218   .2600759    -1.39   0.166     .1416902    1.398961
        3 1  |   .5862065    .349826    -0.89   0.371      .182006     1.88806
        4 1  |   .3903014   .2131491    -1.72   0.085      .133828    1.138291
        5 1  |   .3662498    .198098    -1.86   0.063     .1268755    1.057248
        6 1  |   .3101139    .167915    -2.16   0.031     .1073069    .8962201
        7 1  |   .2630747   .1390217    -2.53   0.012     .0933818    .7411328
        8 1  |   .3516554   .1913706    -1.92   0.055     .1210303     1.02174
        9 1  |   .3277157    .177719    -2.06   0.040     .1132123    .9486384
       10 1  |   .2697093   .1448346    -2.44   0.015     .0941453    .7726688
       11 1  |   .2553095   .1403161    -2.48   0.013     .0869467    .7496882
       12 1  |   .1611979   .0855239    -3.44   0.001      .056984    .4560012
       13 1  |   .1985358    .106346    -3.02   0.003      .069485    .5672656
       14 1  |   .1751922   .0936559    -3.26   0.001     .0614428     .499526
       15 1  |     .28371   .1547531    -2.31   0.021     .0974039    .8263672
       16 1  |   .1739358   .0954922    -3.19   0.001     .0593023    .5101594
       17 1  |   .4822031   .2767176    -1.27   0.204      .156588    1.484915
       18 1  |   .1951106   .1132377    -2.82   0.005     .0625551    .6085534
       19 1  |   .2404992   .1392641    -2.46   0.014     .0773062    .7481913
       20 1  |    .130381   .0752556    -3.53   0.000     .0420632    .4041346
       21 1  |   .4124005   .2567378    -1.42   0.155     .1217329    1.397109
       22 1  |   .1410444   .0845028    -3.27   0.001     .0435896    .4563828
       23 1  |   .5551621   .3857381    -0.85   0.397     .1422288    2.166967
       24 1  |   .1198829   .0786353    -3.23   0.001     .0331455    .4335999
       25 1  |   .3087276   .2258758    -1.61   0.108      .073588    1.295221
       26 1  |   .1533305   .1112956    -2.58   0.010     .0369641    .6360288
       27 1  |   .5476089   .4676354    -0.71   0.481     .1027022    2.919856
       28 1  |   .3554775    .300214    -1.22   0.221     .0679111    1.860731
       29 1  |   1.720054   1.667049     0.56   0.576     .2573775    11.49512
       30 1  |          1  (empty)
             |
       _cons |   .0067619   .0022616   -14.94   0.000     .0035106    .0130244



.  logistic UOF15 c.ICS##i.EverInc

Logistic regression                             Number of obs     =     35,923
                                                LR chi2(3)        =    3662.47
                                                Prob > chi2       =     0.0000
Log likelihood = -6121.6553                     Pseudo R2         =     0.2303

-------------------------------------------------------------------------------
        UOF15 | Odds Ratio   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
          ICS |   1.275128   .0083021    37.33   0.000      1.25896    1.291504
    1.EverInc |   1.122231   .1254018     1.03   0.302      .901501    1.397006
              |
EverInc#c.ICS |
           1  |   .9710621   .0080333    -3.55   0.000     .9554442    .9869354
              |
        _cons |   .0060653    .000495   -62.55   0.000     .0051687    .0071173
-------------------------------------------------------------------------------

Thanky ou so much!

[Clyde Schechter]

Looking at the EverInc variable's results by itself (in either model) is not useful. In either case, it tells you only about the effect of EverInc conditional on ICS = 0. So it is a very incomplete depiction of the role of EverInc. In models with interaction terms, there is no such thing as "the" effect of EverInc. There are, by definition, different effects of EverInc at different levels of ICS. It is simply not possible in these models to assess the effect of EverInc without regard to the value of ICS.

I think to better understand your data, you should look at some margins and plots of margins. I will illustrate the code for the model using continuous ICS, here because I think using 30 different discrete values is too unwieldy to really give understandable results. (When I referred earlier to a discretized ICS, I was under the impression that you had 3 categories: high medium and low. While I generally discourage the use of categorized versions of discrete variables, because they discard information, at least a 3 category version is manageable, and you can wrap your mind around the results.)

margins EverInc, at(ICS = (0(2)30))
marginsplot

[Marvin Aliaga]

Thank you Clyde Schechter . You are right the ICS was originally a continuous variable (from -4 to 35) but I group the values in 4 groups (low groups are less dangerous). I think that my results changed a little bit in the models I showed in #4 in part because I removed some negative ICS values. For the model below, I used the ordinal ICS variable (4 groups). Now, the EverInc variable overall is significant. and all the interaction terms are all significant.

. logistic UOF15 i.ICSC##i.EverIn

Logistic regression                             Number of obs     =     37,588
                                                LR chi2(7)        =    3394.22
                                                Prob > chi2       =     0.0000
Log likelihood = -6499.7206                     Pseudo R2         =     0.2070

------------------------------------------------------------------------------
       UOF15 | Odds Ratio   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        ICSC |
          2  |   2.498961   .2810226     8.14   0.000     2.004643    3.115172
          3  |   13.98179   1.634961    22.56   0.000     11.11802    17.58321
          4  |   55.29607   6.393116    34.71   0.000     44.08407    69.35964
             |
   1.EverInc |   1.323546   .1667028     2.23   0.026     1.034019     1.69414
             |
ICSC#EverInc |
        2 1  |   .7221326   .1153751    -2.04   0.042     .5279838    .9876735
        3 1  |   .4653305    .074398    -4.78   0.000     .3401493    .6365806
        4 1  |   .4981748    .077346    -4.49   0.000     .3674729    .6753644
             |
       _cons |    .013472    .001163   -49.89   0.000      .011375    .0159556
------------------------------------------------------------------------------
 margins EverInc, at(ICSC = (1(1)4))

I. What can we interpret the significant result of overall EverInc? I was expecting PastInc not to be significant since "there different effects of EverInc at different levels of ICS".

Does these sentences look valid to you?
1. Patients in higher ICS groups have greater odd of being involved in UOF incidents compared to those in lower ICS groups.
2.Overall, patients with a previous admission are more likely to be involved in UOF. However, the effect of PastInc is influenced by ICS. As ICS increase, the effect of PastInc on UOF reduce. That is, among patients with high ICS, those with a Past admission are less likely to be involved in UOF compare to those with a no previous admissions.
3. Regarding marginplots: When I do margins EverInc, at(ICSC = (1(1)4)) and then marinplot., am I including the interaction effect here? Do I need to include the interaction in the marigins command term as well? I am sorry for my ignorance.

Thank so much!!

[Clyde Schechter]

What can we interpret the significant result of overall EverInc?

It tells you that in ICS Group 1 (the omitted group), there is a statistically significant effect of EverInc.

With regards to questions 1-3, I am a little bit confused. You refer to PastInc effects, but there is no such variable in your model. Is PastInc the same thing as EverInc? In my answers, I assume it is.

1. Correct.

2. You've got it backwards here. Those with EverInc = 1 are less likely to be involved in incidents, at least given the same classification score. As for how ICS affects the effect of EverInc, it gets a little confusing because the effect of EverInc1 is in the negative direction. So as ICS increases, the difference in probability of UOF incidents between those with EverInc = 1 and those with EverInc = 0 becomes larger. This is because at high levels of ICS, EverInc = 1 is associated with an even greater reduction in the probability of UOF incidents.

3. Because you used factor-variable notation in your logistic regression command, -margins- knows about and takes into account the interaction term.

By the way, since you decided to go to your 4 category variable, you could have written the margins command more simply as:

margins ICSC#EverInc

The use of the at() option is intended primarily for continuous valued variables. The results will be the same when margins are being calculated, but they can differ when marginal effects are calculated.

[Marvin Aliaga]

Sorry about the type. yes- PastInc is EverInc. I am so sorry to get back to you on this but I am a little confuse.

What can we interpret the significant result of overall EverInc?
"It tells you that in ICS Group 1 (the omitted group), there is a statistically significant effect of EverInc"
But EverInc 1=Yes 0=no. The coeffiecient for the EverInc is 1.3. So those EverInc1 are more likely to be involved in UOF than EverInc0 (since Odds ratio is greater than 1. The reference group here is PastInc0. Am I missing something here?

What do you think about the idea of run the model by I the Score categories (4 models). To see how the effect of PastInc on UOF change (even changed directions)?

[Clyde Schechter]

Marvin, there is no need to run the model separately by ICS group. You can get the effect of EverInc in each category with:

margins ICSC, dydx(EverInc)

But EverInc 1=Yes 0=no. The coeffiecient for the EverInc is 1.3. So those EverInc1 are more likely to be involved in UOF than EverInc0 (since Odds ratio is greater than 1. The reference group here is PastInc0. Am I missing something here?

That's right, but only when ICSC = 1. The coefficient of EverInc itself is not the overall effect of EverInc. In fact, there is no such thingas the overall effect of EverInc in a model with interactions. There is a separate effect of EverInc at each level of ICSC. The -margins- command shown above in this post will give all four them to you. You will find that it goes the opposite direction in ICSC Group 1 compared to ICSC Groups 2-4. You can also see that in your graph in #6: the red line is above the blue at ICSC = 1, but it is below the blue at ISCS = 2, 3, 4.

If you are compelled, for some reason, to get a single summary statistic that is related to the effects of EverInc, you can run this:

margins, dydx(EverInc)

That will give you the average effect of EverInc in your data set. It will not generalize to any other situation, and, particularly since EverInc has a reducing effect in one ICSC group and an increasing effect in the others, I think that any single summary statistic obscures the reality of what's going on. So I don't recommend doing this, but if somebody is pressing you for a single number, this one is less bad than any other I can think of.

[Marvin Aliaga]

Thank you so much! I will look deeper into interaction effects.