Comparison of models using xtlogit - different rho

[Rodrigo Primor]

Hi,

I'm using a database with bot firm specific (financial) and macroeconomic variables (annual data). I estimated two models using xtlogit, re because the data is cross-sectional (several companies and 10 years). One of the models contains only the financial variables (model 1) and the other contains both financial and macroeconomic variables (model 2). The dependent variable assumes 1 if the company filed for bankruptcy in one year and 0 if the company is active.

I am having a problem with LR test of rho=0. In model 1 the null hypothesis is rejected but when i introduce macroeconomic variables (model 2), the null hypothesis is not rejected.

My question is: Can i still use model 2 or it doesn't make sense to compare it with model 1?

The code of the two estimates is below.

Model 1

Random effects u_i ~ Gaussian                   Obs per group:
                                                              min =          1
                                                              avg =        7.3
                                                              max =          9

Integration method: mvaghermite                 Integration pts.  =         12

                                                Wald chi2(25)     =     823.70
Log likelihood  = -1878.7941                    Prob > chi2       =     0.0000

------------------------------------------------------------------------------
      status |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
          V1 |  -3.589235   .8570724    -4.19   0.000    -5.269066   -1.909404
          V2 |  -.4279108   .0823644    -5.20   0.000    -.5893421   -.2664795
          V4 |   1.996244   .2673317     7.47   0.000     1.472284    2.520205
          V5 |  -5.072776   .6807636    -7.45   0.000    -6.407048   -3.738504
          V6 |  -.0425941   .2253222    -0.19   0.850    -.4842174    .3990292
          V7 |  -.1452252   .0502684    -2.89   0.004    -.2437494   -.0467011
         V11 |   .0183279    .004647     3.94   0.000     .0092199    .0274359
         V15 |   1.393574    .218075     6.39   0.000     .9661551    1.820993
         V16 |    .001962   .0012059     1.63   0.104    -.0004016    .0043255
         V17 |   1.184356   .2088272     5.67   0.000     .7750622     1.59365
             |
      sector |
          2  |   .2556492   .6723397     0.38   0.704    -1.062112    1.573411
          3  |   .2736469    .707907     0.39   0.699    -1.113825    1.661119
          4  |   .9151518    .458941     1.99   0.046      .015644     1.81466
          5  |    1.01388   .2034456     4.98   0.000     .6151342    1.412626
          6  |   .1233476   .2127867     0.58   0.562    -.2937068    .5404019
          7  |  -.2620406   .3344664    -0.78   0.433    -.9175827    .3935016
          8  |   .2045674   .3736777     0.55   0.584    -.5278274    .9369623
          9  |  -1.463543   .4967248    -2.95   0.003    -2.437105     -.48998
         10  |  -.4378541   .3124299    -1.40   0.161    -1.050205    .1744972
         11  |  -.0301741   .2484104    -0.12   0.903    -.5170495    .4567013
         12  |  -.5312104   .2595223    -2.05   0.041    -1.039865   -.0225561
         13  |  -.4360059   .3613646    -1.21   0.228    -1.144267    .2722557
         14  |          0  (empty)
         15  |  -.8903856   .7502058    -1.19   0.235    -2.360762    .5799908
         16  |  -.5424833   .5780727    -0.94   0.348    -1.675485    .5905183
         17  |  -.9735522    1.14049    -0.85   0.393    -3.208872    1.261768
             |
       _cons |  -8.597101   .7130553   -12.06   0.000    -9.994664   -7.199538
-------------+----------------------------------------------------------------
    /lnsig2u |   .1491919    .422952                     -.6797788    .9781627
-------------+----------------------------------------------------------------
     sigma_u |   1.077449   .2278546                       .711849    1.630817
         rho |   .2608307   .0815443                      .1334693    .4470283
------------------------------------------------------------------------------
LR test of rho=0: chibar2(01) = 6.90                   Prob >= chibar2 = 0.004

Model 2

Random-effects logistic regression              Number of obs     =    237,724
Group variable: ID                              Number of groups  =     31,607

Random effects u_i ~ Gaussian                   Obs per group:
                                                              min =          1
                                                              avg =        7.5
                                                              max =          9

Integration method: mvaghermite                 Integration pts.  =         12

                                                Wald chi2(37)     =    1756.42
Log likelihood  = -1619.5995                    Prob > chi2       =     0.0000

------------------------------------------------------------------------------
      status |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
          V1 |  -3.863255   .9183819    -4.21   0.000    -5.663251    -2.06326
          V2 |  -.2761301   .0862837    -3.20   0.001     -.445243   -.1070172
          V4 |   2.048448   .2672627     7.66   0.000     1.524623    2.572273
          V5 |  -4.498262   .7261031    -6.20   0.000    -5.921398   -3.075126
          V7 |  -.1694759   .0568057    -2.98   0.003     -.280813   -.0581387
         V11 |   .0195027   .0047237     4.13   0.000     .0102445    .0287609
         V15 |   1.290578   .2253473     5.73   0.000     .8489054    1.732251
         V16 |    .002377    .001195     1.99   0.047     .0000349     .004719
         V17 |   .6493138   .2148664     3.02   0.003     .2281833    1.070444
          M2 |  -.5574164    .093577    -5.96   0.000    -.7408241   -.3740088
          M4 |   .2295884   .0824058     2.79   0.005     .0680761    .3911008
          M5 |   .1142688   .0334717     3.41   0.001     .0486654    .1798722
          M6 |   .0213722   .0102355     2.09   0.037     .0013111    .0414334
         M11 |  -3.515599   .8124386    -4.33   0.000     -5.10795   -1.923249
         M12 |  -.0002193   .0000574    -3.82   0.000    -.0003317   -.0001068
             |
      sector |
          2  |   .3550126   .6985123     0.51   0.611    -1.014046    1.724071
          3  |   .5429847   .7310161     0.74   0.458    -.8897806     1.97575
          4  |   .7643641   .4636417     1.65   0.099    -.1443569    1.673085
          5  |   1.183401   .1948808     6.07   0.000     .8014418     1.56536
          6  |   .3518076   .2103622     1.67   0.094    -.0604949      .76411
          7  |   -.165129   .3255784    -0.51   0.612     -.803251    .4729929
          8  |   .7142812   .3585762     1.99   0.046     .0114848    1.417078
          9  |  -.8113489   .4836383    -1.68   0.093    -1.759263    .1365648
         10  |    .235248   .3107224     0.76   0.449    -.3737567    .8442527
         11  |   .0266005   .2444878     0.11   0.913    -.4525867    .5057877
         12  |  -.1335239   .2502774    -0.53   0.594    -.6240586    .3570108
         13  |   .0704027   .3491813     0.20   0.840    -.6139802    .7547855
         14  |          0  (empty)
         15  |  -.3887461   .7328763    -0.53   0.596    -1.825157    1.047665
         16  |  -.1484486   .5398745    -0.27   0.783    -1.206583     .909686
         17  |  -.6588616   1.092302    -0.60   0.546    -2.799735    1.482011
             |
     country |
          2  |          0  (empty)
          3  |  -1.163774    .560085    -2.08   0.038    -2.261521    -.066028
          4  |  -.4278757   .6171485    -0.69   0.488    -1.637465    .7817132
          5  |   -1.17591   .7149728    -1.64   0.100    -2.577231    .2254108
          6  |  -1.073482   .4042748    -2.66   0.008    -1.865846   -.2811179
          7  |          0  (empty)
          9  |   3.712456   .7054062     5.26   0.000     2.329885    5.095027
         10  |  -3.306783   1.314683    -2.52   0.012    -5.883514   -.7300511
         11  |          0  (empty)
         12  |  -.0746633   .4470131    -0.17   0.867    -.9507928    .8014662
             |
       _cons |  -14.66771   3.649871    -4.02   0.000    -21.82133   -7.514098
-------------+----------------------------------------------------------------
    /lnsig2u |  -8.701804   12.62254                     -33.44153    16.03792
-------------+----------------------------------------------------------------
     sigma_u |   .0128952    .081385                      5.47e-08     3038.02
         rho |   .0000505   .0006379                      9.11e-16    .9999996
------------------------------------------------------------------------------
LR test of rho=0: chibar2(01) = 3.2e-04                Prob >= chibar2 = 0.493

Many thanks!

[Carlo Lazzaro]

Rodrigo:
the main issue with your data is not rejecting (or not) the null under -rho=0- but which one of those model give the truest and fairest view of the data generating process.

[Rodrigo Primor]

Thanks Carlo, i was worried about that because i thought that if the panel-level variance component is not significant, the model could not be used and compared with the other.

Best regards

[Carlo Lazzaro]

Rodrigo:
I understand your concern.
That said:
- I would check possible variations in the number of observations between the two models due to missing data (you do not share the first lines of your codes; hence, it difficult to reply any further on this topic);
- I would consider using -testparm- to investigate whether some of your categorical predictors are, actually, redundant (both models).;
- eventually, I woud re-run both models amended when necessary;
As an aside, I would say that obtaining different -rho- with different models is not surprising; besides, in the second models you seemingly have more collinearity problems than in the first one.

[Clyde Schechter]

I would just amplify Carlo's good advice. The shrinkage of rho to near 0 in the second model simply means that the additional country variables themselves account for essentially all of the variation that the random effects explained in the first model. This by no means invalidates the model. It's just saying that with all the information you have provided, the random effects have no work left to do.

If you wanted to, you could replace the second model with a simple -logit- regression and you would get essentially the same results. But if one of the things you want to do is contrast the two models with a likelihood ratio test, I wouldn't do that: stick with -xtlogit, re- for both if you want to contrast them. Also, if you want to contrast the two models, then you must first assure that they use the same estimation sample. If there are any observations having missing values of the added country-level variables that were part of the estimation sample in the first model, they will drop out in the second one, so the comparison would not be valid. The simplest way to assure you are using the same estimation sample is to run the larger model first, and then run the smaller model with the condition -if e(sample)-. That will assure that the smaller model doesn't grab any observations that are ineligible for the larger one.

[Rodrigo Primor]

Sorry for the late reply but i could not answer during the weekend. Now i am using xtlogit, re to compare the two models with likelihood ratio test like you said. I've also identified the variables that were causing the rho "problem". The if e(sample) condition was very helpful.

Thank you Carlo and Clyde.

Best regards.