ologit and brant test

[Paolo Velasquez]

Hello everyone,

I have conducted a brant test after an ordered logistic regression in order to test for the parallel regression assumption. My main independent variable is foreign_bakgrnd which does not seem to violate the assumption. However, I'm worried about the significant value of all the variables, i.e. "All" with a p<0.05. Should I consider doing mlogit instead or is ologit better in this case? I also conducted other tests which seem to point to ordered logistic regression as the best fit. However, I'm not 100% sure.

Some background information: The dependent variable ranges from 1 to 5 "very good, good, neither good nor bad, bad, very bad," to a proposal "Accept fewer refugees?". Foreign_bakgrnd is the percentage of immigrants in each municipality. There are 50 municipalities(kommun).





Ologit

ologit refugee  gender age educ income student unemp foreign_bakgrnd tax total_unemp welfare  if raised_swe==1 & mom==1 & dad==1 & f
> 80a==1 & citizen==1, cluster (kommun)

Iteration 0:   log pseudolikelihood = -3262.2904  
Iteration 1:   log pseudolikelihood = -3140.9864  
Iteration 2:   log pseudolikelihood = -3140.2161  
Iteration 3:   log pseudolikelihood = -3140.2157  
Iteration 4:   log pseudolikelihood = -3140.2157  

Ordered logistic regression                       Number of obs   =       2035
                                                  Wald chi2(10)   =     313.20
                                                  Prob > chi2     =     0.0000
Log pseudolikelihood = -3140.2157                 Pseudo R2       =     0.0374

                                   (Std. Err. adjusted for 50 clusters in kommun)
---------------------------------------------------------------------------------
                |               Robust
        refugee |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
----------------+----------------------------------------------------------------
         gender |   .3089709    .070089     4.41   0.000      .171599    .4463427
            age |   .0055836   .0043592     1.28   0.200    -.0029603    .0141275
           educ |    .471975   .0498627     9.47   0.000     .3742459    .5697041
         income |   .0187043   .0176838     1.06   0.290    -.0159553    .0533639
        student |   .7294596    .185481     3.93   0.000     .3659235    1.092996
          unemp |   .2874827   .2435057     1.18   0.238    -.1897796     .764745
foreign_bakgrnd |   .0259353   .0095523     2.72   0.007     .0072131    .0446576
            tax |  -.0011078   .0067755    -0.16   0.870    -.0143875    .0121719
    total_unemp |  -.0617319   .0238923    -2.58   0.010      -.10856   -.0149038
        welfare |   .0026904   .0484414     0.06   0.956    -.0922529    .0976337
----------------+----------------------------------------------------------------
          /cut1 |   .6259649   .7343175                      -.813271    2.065201
          /cut2 |   1.721342    .724796                      .3007678    3.141916
          /cut3 |   2.781114   .7424679                      1.325904    4.236325
          /cut4 |    3.80011   .7355121                      2.358533    5.241687

Mlogit

  mlogit refugee  gender age educ income student unemp foreign_bakgrnd tax total_unemp welfare  if raised_swe==1 & mom==1 & dad==1 & f
> 80a==1 & citizen==1, cluster (kommun)

Iteration 0:   log pseudolikelihood = -3262.2904  
Iteration 1:   log pseudolikelihood = -3118.6774  
Iteration 2:   log pseudolikelihood = -3114.8747  
Iteration 3:   log pseudolikelihood = -3114.8613  
Iteration 4:   log pseudolikelihood = -3114.8613  

Multinomial logistic regression                   Number of obs   =       2035
                                                  Wald chi2(40)   =    1788.08
                                                  Prob > chi2     =     0.0000
Log pseudolikelihood = -3114.8613                 Pseudo R2       =     0.0452

                                        (Std. Err. adjusted for 50 clusters in kommun)
--------------------------------------------------------------------------------------
                     |               Robust
             refugee |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
---------------------+----------------------------------------------------------------
_Very_Good           |
              gender |   -.394604   .1127504    -3.50   0.000    -.6155907   -.1736173
                 age |   -.010347   .0045848    -2.26   0.024     -.019333    -.001361
                educ |  -.3645869   .0721144    -5.06   0.000    -.5059286   -.2232452
              income |  -.0522729   .0287455    -1.82   0.069     -.108613    .0040672
             student |  -.4549115   .3531302    -1.29   0.198    -1.147034    .2372111
               unemp |   .0304258   .3571199     0.09   0.932    -.6695163    .7303679
     foreign_bakgrnd |   .0016004   .0099909     0.16   0.873    -.0179815    .0211823
                 tax |   .0018852   .0105668     0.18   0.858    -.0188254    .0225957
         total_unemp |   .0154779   .0351648     0.44   0.660    -.0534439    .0843997
             welfare |   .0490317   .0599628     0.82   0.414    -.0684932    .1665565
               _cons |   1.632298   1.059698     1.54   0.123    -.4446714    3.709268
---------------------+----------------------------------------------------------------
Fairly_Good          |
              gender |  -.0450976    .121754    -0.37   0.711    -.2837311    .1935359
                 age |  -.0067947   .0046264    -1.47   0.142    -.0158623    .0022729
                educ |  -.2249964   .0519674    -4.33   0.000    -.3268507   -.1231422
              income |  -.0199047   .0257654    -0.77   0.440     -.070404    .0305945
             student |  -.2329129   .3298539    -0.71   0.480    -.8794146    .4135888
               unemp |  -.4127772   .4627189    -0.89   0.372     -1.31969    .4941352
     foreign_bakgrnd |  -.0044638   .0106793    -0.42   0.676    -.0253949    .0164672
                 tax |   .0096151   .0088208     1.09   0.276    -.0076734    .0269036
         total_unemp |    .006274   .0330595     0.19   0.849    -.0585215    .0710695
             welfare |   .0407527   .0541683     0.75   0.452    -.0654151    .1469206
               _cons |  -.0608351   .9518732    -0.06   0.949    -1.926472    1.804802
---------------------+----------------------------------------------------------------
Neither_good_nor_bad |  (base outcome)
---------------------+----------------------------------------------------------------
Fairly_bad           |
              gender |  -.0628752   .1013472    -0.62   0.535    -.2615122    .1357617
                 age |   .0040965   .0060997     0.67   0.502    -.0078586    .0160516
                educ |   .2815299    .084509     3.33   0.001     .1158953    .4471646
              income |   .0234315   .0260612     0.90   0.369    -.0276474    .0745105
             student |   .5616208   .2876051     1.95   0.051     -.002075    1.125316
               unemp |   .0937818   .3774915     0.25   0.804     -.646088    .8336516
     foreign_bakgrnd |   .0112605   .0156116     0.72   0.471    -.0193376    .0418586
                 tax |   .0125842   .0084345     1.49   0.136    -.0039471    .0291155
         total_unemp |  -.0368523   .0329919    -1.12   0.264    -.1015153    .0278107
             welfare |   .0899803   .0785352     1.15   0.252    -.0639458    .2439064
               _cons |  -2.966431   .9481258    -3.13   0.002    -4.824723   -1.108139
---------------------+----------------------------------------------------------------
Very_bad             |
              gender |   .2793758   .1481659     1.89   0.059     -.011024    .5697756
                 age |  -.0066449   .0054863    -1.21   0.226     -.017398    .0041081
                educ |   .4074613   .0708812     5.75   0.000     .2685368    .5463859
              income |  -.0283097    .033077    -0.86   0.392    -.0931394    .0365201
             student |   .5167056   .3865466     1.34   0.181    -.2409118    1.274323
               unemp |   .3544362   .4450767     0.80   0.426    -.5178981    1.226771
     foreign_bakgrnd |   .0511379   .0121621     4.20   0.000     .0273007    .0749752
                 tax |  -.0007457   .0101354    -0.07   0.941    -.0206107    .0191194
         total_unemp |  -.1032037   .0395097    -2.61   0.009    -.1806413   -.0257661
             welfare |   .0197273   .0590026     0.33   0.738    -.0959156    .1353702
               _cons |  -1.737012   1.161006    -1.50   0.135    -4.012543     .538519
--------------------------------------------------------------------------------------

 brant

Brant test of parallel regression assumption

                  |       chi2     p>chi2      df
 -----------------+------------------------------
              All |      50.52      0.011      30
 -----------------+------------------------------
           gender |       4.86      0.182       3
              age |       8.33      0.040       3
             educ |       2.06      0.561       3
           income |       5.84      0.119       3
          student |       1.57      0.666       3
            unemp |       2.25      0.522       3
  foreign_bakgrnd |       5.61      0.132       3
              tax |       1.82      0.611       3
      total_unemp |       1.29      0.732       3
          welfare |       1.13      0.769       3

A significant test statistic provides evidence that the parallel
regression assumption has been violated.

I have also used the oparallel command:

oparallel, ic

Tests of the parallel regression assumption

                 |   Chi2     df  P>Chi2
-----------------+----------------------
     Wolfe Gould |  44.44     30   0.043
           Brant |  50.52     30   0.011
           score |  47.37     30   0.023
likelihood ratio |  45.59     30   0.034
            Wald |  48.76     30   0.017



Information criteria

      |     ologit     gologit  difference
------+------------------------------------
  AIC |    6308.43     6322.84      -14.41
  BIC |    6387.09     6570.05     -182.96

I also compared the saved model (ologit) to the current model (mlogit):

. fitstat, dif force

                         |     Current        Saved   Difference
-------------------------+---------------------------------------
Log-likelihood           |                                      
                   Model |   -3114.861    -3140.216       25.354
          Intercept-only |   -3262.290    -3262.290        0.000
-------------------------+---------------------------------------
Chi-square               |                                      
    D (df=1991/2021/-30) |    6229.723     6280.431      -50.709
      Wald (df=40/10/30) |    1788.084      313.196     1474.888
                 p-value |       0.000        0.000        0.010
-------------------------+---------------------------------------
R2                       |                                      
                McFadden |       0.045        0.037        0.008
     McFadden (adjusted) |       0.032        0.033       -0.001
            Cox-Snell/ML |       0.135        0.113        0.022
  Cragg-Uhler/Nagelkerke |       0.141        0.118        0.023
                   Count |       0.309        0.290        0.019
        Count (adjusted) |       0.096        0.071        0.025
-------------------------+---------------------------------------
IC                       |                                      
                     AIC |    6317.723     6308.431        9.291
        AIC divided by N |       3.105        3.100        0.005
       BIC (df=44/14/30) |    6564.926     6387.087      177.839

Note: Likelihood-ratio test assumes saved model nested in current model.

Difference of  177.839 in BIC provides very strong support for saved model.

[Marcos Almeida]

Hello Paolo,

Welcome to the Stata Forum,

Unfortunately, the model's results are not presented in #1.

I wonder whether you have also tried the user-written gologit2 (R. Williams) particularly with the "autofit" option.

Best,

Marcos

[Paolo Velasquez]

Hi Marcos,

I have edited the post with the models. I have not tried gologit2 but will look into it.

Best,

[Marcos Almeida]

Hello Paolo,

I wish to underline that, as you know, the outcomes under - ologit - are differently interpreted if compared with the outcomes under - mlogit -, more so when we change the base outcome to a "middle ground", as you presented in #1.

This notwithstanding, now that you presented the outputs, it seems there is much if favor of an ordinal logistic regression: higher R-squared, lower AIC and BIC, to name a few.

What is more, under a partial proportional odds models, such as I suggested in #2, I think you may get an appropriate model and, what is more, cope with violations of the PO assumption.

Best,

Marcos

[Richard Williams]

Note that the brant command indicates that only age is a problem. That might make the problem a good candidate for gologit.

However, as Marcos points out, both the BIC and AIC commands favor ologit over gologit (although a partial proportional odds model might get the edge over both).

Just based on what is shown, I would NOT want to go with the far less parsimonious mlogit. I think a good case could be made for ologit; although you can try gologit2 and see if you get a better model still.

Even though I wrote gologit2. I think life is much simpler if you can reasonably justify ologit, e.g. models are easier to interpret. I think noting that only one variable was problematic (and even then not by much) and that BIC and AIC tests favor ologit over the alternatives, I think you could make a reasonable case for ologit.

[LeDuc Hoang]

Hello everyone!
How can I solve problem "not all independent variables can be retained in binary logits[INDENT=2]brant test cannot be computed" when I conduct Brant test for parallel regression assumption. Please help me! Tkss
[/INDENT]

[Richard Williams]

Welcome to Statalist. In general, it is better to start a new thread than to add on to a very old one. Also, see pt #12 of the FAQ on posting questions effectively.

I think I can guess at what is going wrong though. Brant runs a aseries of binary logits, collapsing the ordinal variable each time. So, if there are 4 categories, first it runs category 1 vs categories 2, 3, 4. Then, it runs 1 and 2 versus 3 and 4. Finally, 1, 2, 3 vs 4.

The message is telling you that in at least one binary logit some vars are getting dropped. This might happen if, say, when the dv = 1, one of the ivs is always missing. Or, say, when gender = 1, the dv always equals 1, 2, or 3. Or, it may be that some categories of the dv have very few cases, e.g. only 4 cases of the dv have value 1 and you have 5 variables in the model.

Possible solutions include combining some adjacent categories, e.g. if category 1 only a few cases combine it with category 2.

I would start by just running a frequency of the DV. If the problem isn't obvious, then I would run very simple models, adding 1 variable at a time. Or, maybe just run a bunch of crosstabs between your DV and the IVs. Hopefully you will eventually identify where the problem is.

Also, if you add the detail option to the brant command, you will see the results for the different binary logistic regressions.

Good luck. If problems persist, show your code and output.