Comparing Estimated Coefficients across Samples with Mixed Logit Choice Model

[Sunny Wong]

Dear all,

I need help comparing estimated coefficients across samples with the mixed logit choice model -cmmixlogit-.

I understand that I can compare OLS coefficients (and other regression methods) across samples with seemingly unrelated estimation -suest-.

However, -cmmixlogit- is not supported by -suest-.

May I ask if there is a way to test the equality of estimated non-linear coefficients between two samples with -cmmixlogit-?

Here is an example:


webuse inschoice, clear

cmset id insurance

* The Mixed Logit Regression for the Entire Sample *
cmmixlogit choice premium, random(deductible) coeflegend nocon

* Testing the nonlinear combinations of parameters *
nlcom (ratio: (_b[insurance:premium]/_b[insurance:deductible]))

* Now, I would like to separate the entire sample into two sub-samples *
gen sample = 0
replace sample = 1 if id>=126

** Here is the estimated model for the first Sub-Sample **
cmmixlogit choice premium if sample==0, random(deductible) coeflegend nocon
nlcom (ratio0: (_b[insurance:premium]/_b[insurance:deductible]))

** Here is the estimated model for the second Sub-Sample **
cmmixlogit choice premium if sample==1, random(deductible) coeflegend nocon
nlcom (ratio1: (_b[insurance:premium]/_b[insurance:deductible]))


My question: Is there a way to test if ratio0 = ratio1?

Any suggestions would be greatly appreciated. Thank you all.

Sincerely,

Sunny

[Paul Arnem]

Did you find a solution to this problem?

Kind regards

[Erik Ruzek]

I don't know if what I am about to propose is possible in Stata, but it should be in theory. Could you formulate the model in gsem and use the group() and ginvariant() options to test whether the parameters are the same across the samples? The choice models manual for Stata 19, on page 19, suggests this option for cross-sectional choice data. This google scholar page shows citations of a 2008 paper by Temme et al that demonstrates the idea.

[Joerg Luedicke (StataCorp)]

This is straightforward to do by including interaction terms with the subsample indicator. Starting with OP's example:

. webuse inschoice, clear
(Fictional health insurance data)

. cmset id insurance

     Case ID variable: id
Alternatives variable: insurance

. 
. * The Mixed Logit Regression for the Entire Sample *
. cmmixlogit choice premium, random(deductible) coeflegend nocon

Fitting fixed parameter model:

Fitting full model:

Iteration 0:  Log simulated-likelihood = -312.07312  
Iteration 1:  Log simulated-likelihood = -312.05061  
Iteration 2:  Log simulated-likelihood = -312.05058  

Mixed logit choice model                       Number of obs      =      1,250
Case ID variable: id                           Number of cases    =        250

                                               Alts per case: min =          5
                                                              avg =        5.0
                                                              max =          5
Integration sequence:      Hammersley
Integration points:               567             Wald chi2(2)    =      99.07
Log simulated-likelihood = -312.05058             Prob > chi2     =     0.0000

--------------------------------------------------------------------------------
        choice | Coefficient  Legend
---------------+----------------------------------------------------------------
insurance      |
       premium |  -1.992249  _b[insurance:premium]
    deductible |  -2.168022  _b[insurance:deductible]
---------------+----------------------------------------------------------------
/Normal        |
 sd(deductible)|   .3407646  _b[/Normal:sd(deductible)]
--------------------------------------------------------------------------------
LR test vs. fixed parameters: chibar2(01) =     0.18  Prob >= chibar2 = 0.3339

. 
. * Testing the nonlinear combinations of parameters *
. nlcom (ratio: (_b[insurance:premium]/_b[insurance:deductible]))

       ratio: (_b[insurance:premium]/_b[insurance:deductible])

------------------------------------------------------------------------------
      choice | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
       ratio |   .9189251   .0458289    20.05   0.000     .8291021    1.008748
------------------------------------------------------------------------------

. 
. * Now, I would like to separate the entire sample into two sub-samples *
. gen sample = 0

. replace sample = 1 if id>=126
(625 real changes made)

. 
. ** Here is the estimated model for the first Sub-Sample **
. cmmixlogit choice premium if sample==0, random(deductible) coeflegend nocon

Fitting fixed parameter model:

Fitting full model:

Iteration 0:  Log simulated-likelihood = -142.01591  (not concave)
Iteration 1:  Log simulated-likelihood = -141.80722  
Iteration 2:  Log simulated-likelihood = -141.64042  
Iteration 3:  Log simulated-likelihood = -141.64025  
Iteration 4:  Log simulated-likelihood = -141.64025  

Mixed logit choice model                       Number of obs      =        625
Case ID variable: id                           Number of cases    =        125

                                               Alts per case: min =          5
                                                              avg =        5.0
                                                              max =          5
Integration sequence:      Hammersley
Integration points:               547             Wald chi2(2)    =      52.66
Log simulated-likelihood = -141.64025             Prob > chi2     =     0.0000

--------------------------------------------------------------------------------
        choice | Coefficient  Legend
---------------+----------------------------------------------------------------
insurance      |
       premium |  -2.583841  _b[insurance:premium]
    deductible |   -2.73328  _b[insurance:deductible]
---------------+----------------------------------------------------------------
/Normal        |
 sd(deductible)|   .6575135  _b[/Normal:sd(deductible)]
--------------------------------------------------------------------------------
LR test vs. fixed parameters: chibar2(01) =     1.25  Prob >= chibar2 = 0.1320

. nlcom (ratio0: (_b[insurance:premium]/_b[insurance:deductible]))

      ratio0: (_b[insurance:premium]/_b[insurance:deductible])

------------------------------------------------------------------------------
      choice | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
      ratio0 |   .9453262     .06076    15.56   0.000     .8262387    1.064414
------------------------------------------------------------------------------

. est store m0

. 
. ** Here is the estimated model for the second Sub-Sample **
. cmmixlogit choice premium if sample==1, random(deductible) coeflegend nocon

Fitting fixed parameter model:

Fitting full model:

Iteration 0:  Log simulated-likelihood = -167.64878  
Iteration 1:  Log simulated-likelihood = -167.55997  
Iteration 2:  Log simulated-likelihood = -167.55803  
Iteration 3:  Log simulated-likelihood = -167.55802  

Mixed logit choice model                       Number of obs      =        625
Case ID variable: id                           Number of cases    =        125

                                               Alts per case: min =          5
                                                              avg =        5.0
                                                              max =          5
Integration sequence:      Hammersley
Integration points:               547             Wald chi2(2)    =      52.36
Log simulated-likelihood = -167.55802             Prob > chi2     =     0.0000

--------------------------------------------------------------------------------
        choice | Coefficient  Legend
---------------+----------------------------------------------------------------
insurance      |
       premium |  -1.622254  _b[insurance:premium]
    deductible |  -1.823438  _b[insurance:deductible]
---------------+----------------------------------------------------------------
/Normal        |
 sd(deductible)|   8.13e-06  _b[/Normal:sd(deductible)]
--------------------------------------------------------------------------------
LR test vs. fixed parameters: chibar2(01) =     0.00  Prob >= chibar2 = 0.5000

. nlcom (ratio1: (_b[insurance:premium]/_b[insurance:deductible]))

      ratio1: (_b[insurance:premium]/_b[insurance:deductible])

------------------------------------------------------------------------------
      choice | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
      ratio1 |   .8896676    .067522    13.18   0.000     .7573269    1.022008
------------------------------------------------------------------------------

. est store m1

We can estimate the same parameters including interaction terms:

. // Estimate two sets of coefficients, one for each subsample
. cmmixlogit      choice c.premium#i.sample, random(c.deductible#i.sample) nocon

Fitting fixed parameter model:

Fitting full model:

Iteration 0:  Log simulated-likelihood = -309.66231  (not concave)
Iteration 1:  Log simulated-likelihood = -309.44515  
Iteration 2:  Log simulated-likelihood = -309.20364  
Iteration 3:  Log simulated-likelihood = -309.19824  
Iteration 4:  Log simulated-likelihood = -309.19823  

Mixed logit choice model                       Number of obs      =      1,250
Case ID variable: id                           Number of cases    =        250

                                               Alts per case: min =          5
                                                              avg =        5.0
                                                              max =          5
Integration sequence:      Hammersley
Integration points:               579             Wald chi2(4)    =     105.01
Log simulated-likelihood = -309.19823             Prob > chi2     =     0.0000

-----------------------------------------------------------------------------------------
                 choice | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
------------------------+----------------------------------------------------------------
insurance               |
       sample#c.premium |
                     0  |  -2.583826   .3564593    -7.25   0.000    -3.282474   -1.885179
                     1  |  -1.622269   .2293802    -7.07   0.000    -2.071846   -1.172692
                        |
    sample#c.deductible |
                     0  |  -2.733263    .408346    -6.69   0.000    -3.533606    -1.93292
                     1  |  -1.823462   .2656524    -6.86   0.000    -2.344131   -1.302793
------------------------+----------------------------------------------------------------
/Normal                 |
 sd(0.sample#deductible)|   .6574454   .3515884                      .2304913    1.875275
 sd(1.sample#deductible)|   .0088189    .800133                      5.20e-80    1.50e+75
-----------------------------------------------------------------------------------------
LR test vs. fixed parameters: chi2(2) =       1.25        Prob > chi2 = 0.5358

Note: LR test is conservative and provided only for reference.

. est store m2

And check the results:

. // check results:
. est table m0 m1 m2, se

-----------------------------------------------------
    Variable |     m0           m1           m2      
-------------+---------------------------------------
insurance    |
     premium | -2.5838412    -1.622254               
             |  .35646495     .2293634               
  deductible |   -2.73328   -1.8234384               
             |  .40835421    .26561694               
             |
      sample#|
   c.premium |
          0  |                           -2.5838262  
             |                             .3564593  
          1  |                           -1.6222687  
             |                            .22938022  
             |
      sample#|
c.deductible |
          0  |                            -2.733263  
             |                              .408346  
          1  |                           -1.8234617  
             |                            .26565236  
-------------+---------------------------------------
/Normal      |
sd(deducti~e)|  .65751348    8.127e-06               
             |  .35157944    .80781398               
 sd(0.sample#|
c.deductible)|                            .65744538  
             |                            .35158837  
 sd(1.sample#|
c.deductible)|                            .00881887  
             |                            .80013299  
-----------------------------------------------------
                                         Legend: b/se

The results in the above table show that, with the combined model, we are getting the same estimates as we get with the two separate models (there can be a little bit of numerical noise here due to the use of simulated likelihood). We can also replicate the original ratios:

. // ratio0 and ratio1 from the combined model:
. nlcom (ratio0: _b[insurance:0b.sample#c.premium]/_b[insurance:0b.sample#c.deductible])

      ratio0: _b[insurance:0b.sample#c.premium]/_b[insurance:0b.sample#c.deductible]

------------------------------------------------------------------------------
      choice | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
      ratio0 |   .9453266   .0607594    15.56   0.000     .8262403    1.064413
------------------------------------------------------------------------------

. nlcom (ratio1: _b[insurance:1.sample#c.premium]/_b[insurance:1.sample#c.deductible])

      ratio1: _b[insurance:1.sample#c.premium]/_b[insurance:1.sample#c.deductible]

------------------------------------------------------------------------------
      choice | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
      ratio1 |   .8896642   .0675254    13.18   0.000     .7573169    1.022012
------------------------------------------------------------------------------

And finally, we can test whether ratio0=ratio1:

. // test ratio0=ratio1 (--> ratio1-ratio0=0)
. nlcom (d1v0: _b[insurance:1.sample#c.premium]/_b[insurance:1.sample#c.deductible]- ///
>              _b[insurance:0b.sample#c.premium]/_b[insurance:0b.sample#c.deductible])

        d1v0: _b[insurance:1.sample#c.premium]/_b[insurance:1.sample#c.deductible]-_b[insurance:0b.sample#c.premium]/_b
> [insurance:0b.sample#c.deductible]

------------------------------------------------------------------------------
      choice | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
        d1v0 |  -.0556623   .0908371    -0.61   0.540    -.2336999    .1223752
------------------------------------------------------------------------------