Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Insignificant interaction effects but statistically significant predictive margins after mlogit

    Hello Statalist,

    I am running an mlogit in which the dependent variable is a measure of inter-generational social mobility (1=upward, 2=none, 3=downward) and my main independent variables of interest are sibling exposure (which is continuous and measured as shared person-years) and birth cohort (which is a factor variable with three categories). The model includes an interaction term of these two variables and their base effects, plus other controls and using robust standard errors. The model looks like this:

    mlogit mobility c.sibling_exposure i.cohort c.sibling_exposure#i.cohort x3 x4 x5, vce(robust) baseoutcome(2)

    When I run the model, the interaction of the two variables produces no significant result, but the base effects are significant, suggesting that when exposure=0 there are distinct differences between cohorts, and when the cohort=1, the association of exposure and mobility is significantly different from zero. When I calculate the Average Adjusted Predictions at specific values of exposure as:

    margins cohort, at(sibling_exposure=(0(1)30)) predict(outcome(1))


    the confidence intervals of the predictions for the three cohorts do not overlap at higher levels of exposure (above sibling_exposure=15 roughly), suggesting to me that they are statistically distinct from one another. But I am struggling how to reconcile the difference between confidence intervals of the adjusted predictions and the significance tests of the interaction terms in the mlogit output, and which one is more reasonable to use to interpret the results. Does anyone have any advice? Is my thinking here completely misguided?

    Thanks.

    Joe

  • #2
    I understand your frustration, but I think this sort of thing can happen in a non-linear model. A "classic" reference (in sense of being often mentioned by economists!) is: Ai, C., Norton, E., 2003. Interaction terms in logit and probit models. Economics Letters 80, 123129. They have a Stata Journal article too, I seem to recall. On related issues, see also, "Testing hypotheses about interaction terms in nonlinear models", by William Greene, Economics Letters, 2010, 10.1016/j.econlet.2010.02.014
    In short, I don't think that eyeballing the p-value on your interaction term in a non-linear model is necessarily informative about the statistical significance of cross-partial effects of the sort you may wish to calculate using margins.

    Comment


    • #3
      It would help to see the actual output. Also, you might change your margins command to

      Code:
      margins r.cohort, at(sibling_exposure=(0(1)30)) predict(outcome(1))
      This will give you formal tests of differences.

      Finally, you might consider trying an ologit model, which would be more parsimonious if the ologit assumptions are not violated.
      -------------------------------------------
      Richard Williams, Notre Dame Dept of Sociology
      Stata Version: 17.0 MP (2 processor)

      EMAIL: [email protected]
      WWW: https://www3.nd.edu/~rwilliam

      Comment


      • #4
        Thank you both for the helpful responses so far. Below is the actual output.

        Code:
        . mlogit mobility c.sibling_exposure i.cohort new_birthorder ageatbirth i.illegit centered_yearatt centered_prop i.dadclass c.sibling_exposure#i.cohort i.dadclas
        > s#c.centered_prop if genus=="M" & age_at_enter==0 & age_at_exit>=30 & dadclass<5 & dadclass>1, baseoutcome(2) rrr vce(robust)
        
        Iteration 0:   log pseudolikelihood =  -5978.957  
        Iteration 1:   log pseudolikelihood = -5540.4954  
        Iteration 2:   log pseudolikelihood = -5533.7283  
        Iteration 3:   log pseudolikelihood = -5533.7178  
        Iteration 4:   log pseudolikelihood = -5533.7178  
        
        Multinomial logistic regression                   Number of obs   =       5517
                                                          Wald chi2(28)   =     761.33
                                                          Prob > chi2     =     0.0000
        Log pseudolikelihood = -5533.7178                 Pseudo R2       =     0.0745
        
        -------------------------------------------------------------------------------------------
                                  |               Robust
                         mobility |        RRR   Std. Err.      z    P>|z|     [95% Conf. Interval]
        --------------------------+----------------------------------------------------------------
        1                         |
                 sibling_exposure |   .9912518   .0058706    -1.48   0.138     .9798121    1.002825
                                  |
                           cohort |
                       1884-1890  |    .812711   .1151294    -1.46   0.143     .6156787    1.072798
                       1891-1896  |   .8348731   .1407941    -1.07   0.285     .5998915    1.161898
                                  |
                   new_birthorder |   .9707117   .0194783    -1.48   0.139     .9332758    1.009649
                       ageatbirth |   1.011482   .0068691     1.68   0.093     .9981077    1.025035
                        1.illegit |   .6559465   .0919756    -3.01   0.003     .4983271    .8634206
                 centered_yearatt |    1.05334   .0093829     5.83   0.000     1.035109    1.071891
                    centered_prop |   1.054452   .0602223     0.93   0.353     .9427845    1.179345
                                  |
                         dadclass |
                 Skilled Workers  |   1.386021   1.655739     0.27   0.785     .1333292    14.40835
           Lower Skilled Workers  |   5.215804   3.041626     2.83   0.005     1.663176    16.35702
                                  |
        cohort#c.sibling_exposure |
                       1884-1890  |   1.003959    .007302     0.54   0.587     .9897491    1.018373
                       1891-1896  |   .9832827   .0084961    -1.95   0.051     .9667708    1.000077
                                  |
         dadclass#c.centered_prop |
                 Skilled Workers  |   1.072441   .2465943     0.30   0.761     .6833614    1.683048
           Lower Skilled Workers  |   .8560055   .1396453    -0.95   0.341     .6217488    1.178523
                                  |
                            _cons |   .3977896    .092384    -3.97   0.000     .2523292    .6271038
        --------------------------+----------------------------------------------------------------
        2                         |  (base outcome)
        --------------------------+----------------------------------------------------------------
        3                         |
                 sibling_exposure |   .9950693   .0057047    -0.86   0.389     .9839509    1.006313
                                  |
                           cohort |
                       1884-1890  |   1.000713   .1488479     0.00   0.996     .7476536    1.339427
                       1891-1896  |   1.015284   .1843082     0.08   0.933     .7113218    1.449137
                                  |
                   new_birthorder |    1.03837   .0208673     1.87   0.061     .9982658    1.080085
                       ageatbirth |   1.005843    .007233     0.81   0.418     .9917656    1.020119
                        1.illegit |   1.191251   .1743563     1.20   0.232     .8941661    1.587043
                 centered_yearatt |   .9769435   .0094054    -2.42   0.015     .9586822    .9955527
                    centered_prop |   1.048194   .0576766     0.86   0.392     .9410324    1.167559
                                  |
                         dadclass |
                 Skilled Workers  |     .59571   .6547501    -0.47   0.637      .069099     5.13568
           Lower Skilled Workers  |   .1910623   .1431931    -2.21   0.027      .043978    .8300696
                                  |
        cohort#c.sibling_exposure |
                       1884-1890  |   1.001238   .0072182     0.17   0.864     .9871896    1.015486
                       1891-1896  |   1.000411   .0083434     0.05   0.961     .9841908    1.016898
                                  |
         dadclass#c.centered_prop |
                 Skilled Workers  |   1.055005    .226796     0.25   0.803     .6922621    1.607824
           Lower Skilled Workers  |   .6589927   .1309744    -2.10   0.036     .4463801    .9728735
                                  |
                            _cons |   .5344522   .1285304    -2.61   0.009     .3335826    .8562771
        -------------------------------------------------------------------------------------------
        
        . margins cohort, at(sibling_exposure=(0(1)30)) vsquish noatlegend predict(outcome(1))
        
        Predictive margins                                Number of obs   =       5517
        Model VCE    : Robust
        
        Expression   : Pr(mobility==1), predict(outcome(1))
        
        -------------------------------------------------------------------------------
                      |            Delta-method
                      |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
        --------------+----------------------------------------------------------------
           _at#cohort |
         1#1878-1883  |   .4456164   .0216581    20.58   0.000     .4031673    .4880654
         1#1884-1890  |   .4029082   .0166734    24.16   0.000     .3702289    .4355874
         1#1891-1896  |   .4071273    .022313    18.25   0.000     .3633946      .45086
         2#1878-1883  |   .4442269   .0207584    21.40   0.000     .4035412    .4849125
         2#1884-1890  |   .4022452   .0159331    25.25   0.000     .3710169    .4334735
         2#1891-1896  |   .4023072   .0212616    18.92   0.000     .3606353    .4439791
         3#1878-1883  |   .4428368   .0198817    22.27   0.000     .4038693    .4818043
         3#1884-1890  |   .4015818   .0152094    26.40   0.000      .371772    .4313916
         3#1891-1896  |   .3975022   .0202395    19.64   0.000     .3578334     .437171
         4#1878-1883  |   .4414463   .0190313    23.20   0.000     .4041457    .4787469
         4#1884-1890  |   .4009181   .0145045    27.64   0.000     .3724898    .4293464
         4#1891-1896  |    .392713    .019251    20.40   0.000     .3549818    .4304442
         5#1878-1883  |   .4400552   .0182105    24.16   0.000     .4043633     .475747
         5#1884-1890  |   .4002539   .0138213    28.96   0.000     .3731647    .4273431
         5#1891-1896  |   .3879404   .0183005    21.20   0.000      .352072    .4238087
         6#1878-1883  |   .4386636   .0174234    25.18   0.000     .4045145    .4728128
         6#1884-1890  |   .3995894   .0131629    30.36   0.000     .3737906    .4253882
         6#1891-1896  |   .3831851   .0173934    22.03   0.000     .3490947    .4172755
         7#1878-1883  |   .4372716   .0166745    26.22   0.000     .4045901    .4699531
         7#1884-1890  |   .3989245   .0125331    31.83   0.000     .3743601    .4234889
         7#1891-1896  |   .3784478   .0165354    22.89   0.000      .346039    .4108567
         8#1878-1883  |   .4358791   .0159692    27.29   0.000     .4045801    .4671782
         8#1884-1890  |   .3982592   .0119362    33.37   0.000     .3748646    .4216538
         8#1891-1896  |   .3737293   .0157333    23.75   0.000     .3428927    .4045659
         9#1878-1883  |   .4344862   .0153131    28.37   0.000      .404473    .4644994
         9#1884-1890  |   .3975935   .0113774    34.95   0.000     .3752943    .4198928
         9#1891-1896  |   .3690303   .0149941    24.61   0.000     .3396425    .3984182
        10#1878-1883  |   .4330929   .0147127    29.44   0.000     .4042566    .4619293
        10#1884-1890  |   .3969275   .0108622    36.54   0.000     .3756381    .4182169
        10#1891-1896  |   .3643516   .0143256    25.43   0.000     .3362738    .3924293
        11#1878-1883  |   .4316992   .0141747    30.46   0.000     .4039174    .4594811
        11#1884-1890  |   .3962611   .0103969    38.11   0.000     .3758836    .4166386
        11#1891-1896  |   .3596937    .013736    26.19   0.000     .3327716    .3866158
        12#1878-1883  |   .4303052   .0137062    31.39   0.000     .4034416    .4571688
        12#1884-1890  |   .3955943   .0099883    39.61   0.000     .3760177     .415171
        12#1891-1896  |   .3550574   .0132332    26.83   0.000     .3291208    .3809941
        13#1878-1883  |   .4289108   .0133142    32.21   0.000     .4028154    .4550062
        13#1884-1890  |   .3949272   .0096434    40.95   0.000     .3760266    .4138279
        13#1891-1896  |   .3504435   .0128247    27.33   0.000     .3253074    .3755795
        14#1878-1883  |   .4275161   .0130054    32.87   0.000      .402026    .4530062
        14#1884-1890  |   .3942597    .009369    42.08   0.000     .3758969    .4126225
        14#1891-1896  |   .3458525   .0125168    27.63   0.000     .3213201    .3703848
        15#1878-1883  |   .4261212   .0127854    33.33   0.000     .4010623    .4511801
        15#1884-1890  |   .3935919   .0091711    42.92   0.000     .3756168     .411567
        15#1891-1896  |    .341285   .0123136    27.72   0.000     .3171509    .3654192
        16#1878-1883  |   .4247259   .0126585    33.55   0.000     .3999158    .4495361
        16#1884-1890  |   .3929237   .0090547    43.39   0.000     .3751768    .4106706
        16#1891-1896  |   .3367419   .0122171    27.56   0.000     .3127968    .3606869
        17#1878-1883  |   .4233304   .0126271    33.53   0.000     .3985818    .4480791
        17#1884-1890  |   .3922552   .0090225    43.48   0.000     .3745713     .409939
        17#1891-1896  |   .3322236   .0122262    27.17   0.000     .3082607    .3561866
        18#1878-1883  |   .4219347   .0126915    33.25   0.000     .3970598    .4468097
        18#1884-1890  |   .3915863   .0090753    43.15   0.000      .373799    .4093735
        18#1891-1896  |   .3277309   .0123371    26.56   0.000     .3035506    .3519112
        19#1878-1883  |   .4205388     .01285    32.73   0.000     .3953533    .4457244
        19#1884-1890  |   .3909171   .0092113    42.44   0.000     .3728633    .4089708
        19#1891-1896  |   .3232643   .0125434    25.77   0.000     .2986796    .3478489
        20#1878-1883  |   .4191427   .0130987    32.00   0.000     .3934699    .4448156
        20#1884-1890  |   .3902475   .0094267    41.40   0.000     .3717716    .4087234
        20#1891-1896  |   .3188244   .0128368    24.84   0.000     .2936648     .343984
        21#1878-1883  |   .4177465   .0134321    31.10   0.000     .3914201    .4440729
        21#1884-1890  |   .3895776   .0097159    40.10   0.000     .3705348    .4086204
        21#1891-1896  |   .3144118   .0132077    23.81   0.000     .2885253    .3402984
        22#1878-1883  |   .4163501   .0138438    30.07   0.000     .3892168    .4434834
        22#1884-1890  |   .3889074   .0100724    38.61   0.000     .3691658     .408649
        22#1891-1896  |   .3100271   .0136462    22.72   0.000     .2832812    .3367731
        23#1878-1883  |   .4149537   .0143266    28.96   0.000     .3868741    .4430332
        23#1884-1890  |   .3882369   .0104891    37.01   0.000     .3676786    .4087952
        23#1891-1896  |   .3056709   .0141423    21.61   0.000     .2779524    .3333893
        24#1878-1883  |   .4135571   .0148731    27.81   0.000     .3844063    .4427079
        24#1884-1890  |    .387566    .010959    35.37   0.000     .3660868    .4090452
        24#1891-1896  |   .3013436   .0146868    20.52   0.000      .272558    .3301293
        25#1878-1883  |   .4121605   .0154763    26.63   0.000     .3818275    .4424935
        25#1884-1890  |   .3868948   .0114752    33.72   0.000     .3644039    .4093858
        25#1891-1896  |   .2970459    .015271    19.45   0.000     .2671153    .3269766
        26#1878-1883  |   .4107638   .0161294    25.47   0.000     .3791508    .4423768
        26#1884-1890  |   .3862233   .0120316    32.10   0.000     .3626418    .4098049
        26#1891-1896  |   .2927782   .0158872    18.43   0.000     .2616398    .3239166
        27#1878-1883  |   .4093671   .0168261    24.33   0.000     .3763885    .4423457
        27#1884-1890  |   .3855515   .0126228    30.54   0.000     .3608114    .4102917
        27#1891-1896  |   .2885411   .0165286    17.46   0.000     .2561456    .3209366
        28#1878-1883  |   .4079704    .017561    23.23   0.000     .3735515    .4423894
        28#1884-1890  |   .3848794   .0132438    29.06   0.000     .3589221    .4108368
        28#1891-1896  |    .284335   .0171893    16.54   0.000     .2506446    .3180254
        29#1878-1883  |   .4065738   .0183291    22.18   0.000     .3706495    .4424981
        29#1884-1890  |    .384207   .0138905    27.66   0.000     .3569821    .4114319
        29#1891-1896  |   .2801604   .0178641    15.68   0.000     .2451475    .3151733
        30#1878-1883  |   .4051772   .0191259    21.18   0.000     .3676911    .4426633
        30#1884-1890  |   .3835343   .0145593    26.34   0.000     .3549986      .41207
        30#1891-1896  |   .2760178   .0185485    14.88   0.000     .2396634    .3123722
        31#1878-1883  |   .4037807   .0199477    20.24   0.000     .3646838    .4428775
        31#1884-1890  |   .3828613   .0152472    25.11   0.000     .3529774    .4127451
        31#1891-1896  |   .2719076   .0192387    14.13   0.000     .2342004    .3096149
        -------------------------------------------------------------------------------

        Comment


        • #5
          I'm not clear on what the confusion is. If you are interested in whether differences are significant at all these different points, then use the r.cohort notation as I suggested earlier. But you are calculating 90+ adjusted predictions, and this is for just one of the 3 possible outcomes. I am not sure how useful this is, and in any event it doesn't surprise me if some contrasts across cohorts are significant while others are not.

          Keep in mind too that you aren't getting overall tests of the interaction terms. e.g. you have 4 different interaction terms for cohort and sibling exposure. To test all of them, use testparm, e.g.

          Code:
          webuse nhanes2f, clear
          mlogit health i.female c.age i.female#c.age
          testparm i.female#c.age
          The output for the last command is

          Code:
          . testparm i.female#c.age
          
           ( 1)  [poor]1.female#c.age = 0
           ( 2)  [fair]1.female#c.age = 0
           ( 3)  [average]1o.female#co.age = 0
           ( 4)  [good]1.female#c.age = 0
           ( 5)  [excellent]1.female#c.age = 0
                 Constraint 3 dropped
          
                     chi2(  4) =    6.55
                   Prob > chi2 =    0.1619
          -------------------------------------------
          Richard Williams, Notre Dame Dept of Sociology
          Stata Version: 17.0 MP (2 processor)

          EMAIL: [email protected]
          WWW: https://www3.nd.edu/~rwilliam

          Comment


          • #6
            Thank you both for your help. The proportional odds assumption does not seem to be met (at least based on the Brant test) when I tried the ordered logit. Ultimately, I was interested in the different cohort predictions at different values of exposure, so I think the r.cohort notation is what I was looking for. Thanks again!

            Comment

            Working...
            X