Non-Equivalent Results for a <nestreg: svy: logit> and <svy: logit>?

[David Speed]

Hi Statalist,

I'm running a logit model using svyset. My working assumption was that the final block for a <nestreg: svy: logit> would be identical to a non-blocked <svy: logit>. However, in the nestreg model it ended up kicking out the race variable but in the non-blocked model it was retained.

To be clear, I'm running random (i.e., meaningless) data to ensure that my code executes properly for another project. My primary interest is why the non-equivalency emerged and whether it's something I need to test for in future projects.

I ended up catching this because I was specifically interested in the <trans> variable.

. svy: logit new_acc_005 $group $covariates
(running logit on estimation sample)

BRR replications (1000)
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5 
..................................................    50
..................................................   100
..................................................   150
..................................................   200
..................................................   250
..................................................   300
..................................................   350
..................................................   400
..................................................   450
..................................................   500
..................................................   550
..................................................   600
..................................................   650
..................................................   700
..................................................   750
..................................................   800
..................................................   850
..................................................   900
..................................................   950
..................................................  1000

note: race != 3 predicts failure perfectly
      race dropped and 4 obs not used

Survey: Logistic regression                     Number of obs     =        399
                                                Population size   = 935.167756
                                                Replications      =      1,000
                                                Design df         =      1,000
                                                F(  15,    986)   =       1.97
                                                Prob > F          =     0.0145

------------------------------------------------------------------------------
             |                BRR *
 new_acc_005 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       trans |  -.1759338   1.145994    -0.15   0.878    -2.424762    2.072895
         sex |   .5387569   1.109846     0.49   0.627    -1.639137    2.716651
         age |   .0027263   .0065277     0.42   0.676    -.0100832    .0155358
         lgb |    -.13225   .1253307    -1.06   0.292    -.3781913    .1136913
       dreg2 |  -.1004513   .2031402    -0.49   0.621    -.4990813    .2981787
       dreg3 |   .0954237   .2092463     0.46   0.648    -.3151884    .5060358
       dreg4 |  -.1425171   .3434391    -0.41   0.678    -.8164611    .5314269
       dreg5 |   .0116557   .2155611     0.05   0.957    -.4113482    .4346596
       dreg6 |  -.5883021   .4049385    -1.45   0.147    -1.382929    .2063246
      deduc2 |    -.05574   .1437736    -0.39   0.698    -.3378725    .2263925
      deduc3 |  -.0733822   .2016822    -0.36   0.716    -.4691511    .3223867
        race |          0  (omitted)
       dbmi1 |   -.024893   .2165706    -0.11   0.909     -.449878    .4000919
       dbmi3 |  -.1846642   .1961027    -0.94   0.347    -.5694842    .2001559
       dbmi4 |  -.4033248   .1922317    -2.10   0.036    -.7805486   -.0261009
         inc |  -.0065554    .019577    -0.33   0.738    -.0449721    .0318613
       _cons |   .2090892   .2961285     0.71   0.480    -.3720154    .7901938
------------------------------------------------------------------------------

. nestreg: svy: logit new_acc_005 ($covariates) ($group)
note: race dropped because of estimability
note: o.race dropped because of estimability

Block  1: sex age lgb dreg2 dreg3 dreg4 dreg5 dreg6 deduc2 deduc3 dbmi1 dbmi3 dbmi4 inc
(running logit on estimation sample)

BRR replications (1000)
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5 
..................................................    50
..................................................   100
..................................................   150
..................................................   200
..................................................   250
..................................................   300
..................................................   350
..................................................   400
..................................................   450
..................................................   500
..................................................   550
..................................................   600
..................................................   650
..................................................   700
..................................................   750
..................................................   800
..................................................   850
..................................................   900
..................................................   950
..................................................  1000

Survey: Logistic regression                     Number of obs     =        403
                                                Population size   = 946.549165
                                                Replications      =      1,000
                                                Design df         =      1,000
                                                F(  14,    987)   =       2.02
                                                Prob > F          =     0.0139

------------------------------------------------------------------------------
             |                BRR *
 new_acc_005 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         sex |    .326018   .1192416     2.73   0.006     .0920256    .5600104
         age |   .0022885    .006466     0.35   0.723    -.0104001    .0149771
         lgb |  -.1589832   .1267548    -1.25   0.210    -.4077191    .0897526
       dreg2 |   -.076847    .210202    -0.37   0.715    -.4893346    .3356407
       dreg3 |   .0427959   .2396131     0.18   0.858    -.4274063     .512998
       dreg4 |  -.1621691   .3660137    -0.44   0.658    -.8804121    .5560738
       dreg5 |   .0074862   .2124747     0.04   0.972    -.4094613    .4244337
       dreg6 |  -.5620513   .4126888    -1.36   0.174    -1.371887    .2477841
      deduc2 |  -.1048962   .1425214    -0.74   0.462    -.3845715    .1747792
      deduc3 |  -.1121259   .2031567    -0.55   0.581    -.5107882    .2865365
       dbmi1 |   .0050511   .2095763     0.02   0.981    -.4062088    .4163109
       dbmi3 |  -.2158093   .2265647    -0.95   0.341    -.6604061    .2287875
       dbmi4 |  -.4292535   .1811346    -2.37   0.018    -.7847009    -.073806
         inc |  -.0011619   .0194873    -0.06   0.952    -.0394026    .0370788
       _cons |   .2534628   .2992156     0.85   0.397    -.3336997    .8406252
------------------------------------------------------------------------------

Block  2: trans
(running logit on estimation sample)

BRR replications (1000)
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5 
..................................................    50
..................................................   100
..................................................   150
..................................................   200
..................................................   250
..................................................   300
..................................................   350
..................................................   400
..................................................   450
..................................................   500
..................................................   550
..................................................   600
..................................................   650
..................................................   700
..................................................   750
..................................................   800
..................................................   850
..................................................   900
..................................................   950
..................................................  1000

Survey: Logistic regression                     Number of obs     =        403
                                                Population size   = 946.549165
                                                Replications      =      1,000
                                                Design df         =      1,000
                                                F(  15,    986)   =       1.92
                                                Prob > F          =     0.0181

------------------------------------------------------------------------------
             |                BRR *
 new_acc_005 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         sex |   .4873062   1.089298     0.45   0.655    -1.650266    2.624879
         age |   .0023628   .0066682     0.35   0.723    -.0107224     .015448
         lgb |  -.1577184   .1257229    -1.25   0.210    -.4044294    .0889926
       dreg2 |  -.0763008   .2051487    -0.37   0.710    -.4788721    .3262705
       dreg3 |   .0472102   .2271976     0.21   0.835    -.3986286     .493049
       dreg4 |  -.1614056   .3623199    -0.45   0.656    -.8724002    .5495889
       dreg5 |   .0066607   .2130777     0.03   0.975      -.41147    .4247914
       dreg6 |   -.563992   .4060925    -1.39   0.165    -1.360883    .2328992
      deduc2 |   -.108043   .1467737    -0.74   0.462    -.3960628    .1799768
      deduc3 |  -.1143768   .2161698    -0.53   0.597    -.5385754    .3098217
       dbmi1 |   .0026271   .2140528     0.01   0.990     -.417417    .4226712
       dbmi3 |  -.2122906    .211283    -1.00   0.315    -.6268996    .2023184
       dbmi4 |   -.426041   .1876856    -2.27   0.023    -.7943439   -.0577381
         inc |  -.0007558    .019577    -0.04   0.969    -.0391726    .0376609
       trans |  -.1657635   1.136817    -0.15   0.884    -2.396584    2.065057
       _cons |   .2496266   .2960611     0.84   0.399    -.3313457    .8305988
------------------------------------------------------------------------------


  +-------------------------------------------+
  |       |          Block    Design          |
  | Block |       F     df        df   Pr > F |
  |-------+-----------------------------------|
  |     1 |    2.02     14      1000   0.0139 |
  |     2 |    0.02      1      1000   0.8841 |
  +-------------------------------------------+

Thanks so much!

Cheers,

David.

[Ken Chui]

So, are you asking why the first output and the last output differ? Did you check why they have i) different sample sizes and ii) different lineup of independent variables? The inclusion of race in the first model seems to have omitted 4 cases.

[David Speed]

Ken Chui No, I understand that.

To clarify, I'm asking why it would run fine in one case, but not run fine in an 'identical' case with the exact same specifications of variables. You can see that the same macros $covariates and $group are used in both cases. Race was included in both models, but only booted in one. The models were specified in the same way, except one was a nested model.

Analogously:

regress y x1 x2
nestreg: regress y (x1) (x2)

Would, I assume, produce identical results and have identical observations. Except in the case listed initially, it runs it fine one way, but not the other way. This is what I'm confused about. Basically, if there's relevant multicollinearity then why didn't Stata 'complain' in both cases.