Margins command "not estimable"

[Nicolas Orgeira]

Hi,

I a running a conjoint experiment and would like to test for heterogenous effects by interacting dichotomized survey variables (here satisfied vs not satisfied with public services) with conjoint attributes. Unfortunately, when running the margins command, most of the output can't be computed ("Not estimable"), see below the commands, followed by the (relevant) multi-level logit output, then the margins output. This issue comes up irrespective of the dichotomized variable I'm using, that is irrespective of its average (ranging from 0.05 to 0.9). Any suggestion would be very much welcome!

local cov1 i.gender i.head_of_household c.log_hh_size c.log_age i.school i.main_source c.poverty i.property_value
melogit rating2 i.public_service_satisf##(i.tax_amount ib2.start_provision i.who_to_pay i.how_to_pay i.perks i.town##i.subgroup `cov1') || participant_id:
eststo a_public_service_satisf: margins, dydx(tax_amount start_provision who_to_pay how_to_pay perks) at(public_service_satisf=(0 1)) post noatlegend vsquish

Mixed-effects logistic regression               Number of obs     =     14,289
Group variable: participant_id                  Number of groups  =      2,384

                                                Obs per group:
                                                              min =          4
                                                              avg =        6.0
                                                              max =          6

Integration method: mvaghermite                 Integration pts.  =          7

                                                Wald chi2(84)     =     378.98
Log likelihood = -8896.0602                     Prob > chi2       =     0.0000
----------------------------------------------------------------------------------------------------------
                                 rating2 | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-----------------------------------------+----------------------------------------------------------------
                   public_service_satisf |
                              Satisfied  |   .0259363     1.0731     0.02   0.981    -2.077301    2.129173
                                         |
                              tax_amount |
                                 Medium  |  -.5374263   .1228267    -4.38   0.000    -.7781622   -.2966903
                                   High  |  -.7700518   .1213668    -6.34   0.000    -1.007926   -.5321773
                                         |
                         start_provision |
                        6 months before  |   .0021497   .1206165     0.02   0.986    -.2342543    .2385537
                         6 months after  |  -.2991037   .1204549    -2.48   0.013    -.5351909   -.0630166
                                         |
                              who_to_pay |
                                  Chief  |  -.1169149    .097738    -1.20   0.232     -.308478    .0746481
                                         |
                              how_to_pay |
                       Someone collects  |   .1651372   .1383391     1.19   0.233    -.1060024    .4362768
                                   USSD  |  -.1191946   .1398812    -0.85   0.394    -.3933567    .1549675
                             Mobile app  |   .0062795    .140295     0.04   0.964    -.2686935    .2812526
                                         |
                                   perks |
                       Submit complaint  |  -.1357918   .1196134    -1.14   0.256    -.3702298    .0986461
                      Vote for services  |   -.095936   .1196823    -0.80   0.423     -.330509    .1386369
                                         |
        public_service_satisf#tax_amount |
                       Satisfied#Medium  |   .1387765   .1340347     1.04   0.300    -.1239267    .4014797
                         Satisfied#High  |   .1746668   .1324968     1.32   0.187    -.0850221    .4343557
                                         |
   public_service_satisf#start_provision |
              Satisfied#6 months before  |  -.0275204   .1319488    -0.21   0.835    -.2861353    .2310946
               Satisfied#6 months after  |   .1664973   .1315453     1.27   0.206    -.0913268    .4243213
                                         |
        public_service_satisf#who_to_pay |
                        Satisfied#Chief  |    .076321   .1069467     0.71   0.475    -.1332906    .2859326
                                         |
        public_service_satisf#how_to_pay |
             Satisfied#Someone collects  |  -.2127704   .1513052    -1.41   0.160    -.5093231    .0837823
                         Satisfied#USSD  |   .0838432   .1526913     0.55   0.583    -.2154263    .3831127
                   Satisfied#Mobile app  |  -.0129282    .153263    -0.08   0.933    -.3133182    .2874618
                                         |
             public_service_satisf#perks |
             Satisfied#Submit complaint  |  -.0242847   .1308269    -0.19   0.853    -.2807007    .2321312
            Satisfied#Vote for services  |  -.0270495   .1310077    -0.21   0.836    -.2838199     .229721
                                         |
              public_service_satisf#town |
                        Satisfied#Mansa  |    .432457   .5305448     0.82   0.415    -.6073916    1.472306
                       Satisfied#Samfya  |  -.1635377   .5408718    -0.30   0.762    -1.223627    .8965515
                                         |
          public_service_satisf#subgroup |
                Satisfied#Non-compliant  |  -.0466211   .4815389    -0.10   0.923      -.99042    .8971778
                     Satisfied#Informal  |  -.6170218   .5033722    -1.23   0.220    -1.603613    .3695696
                    Satisfied#Customary  |  -.4494704   .5056163    -0.89   0.374     -1.44046    .5415194
                                         |
     public_service_satisf#town#subgroup |
          Satisfied#Mansa#Non-compliant  |   .2895277   .6365689     0.45   0.649    -.9581244     1.53718
               Satisfied#Mansa#Informal  |  -.8884716   .6292342    -1.41   0.158    -2.121748    .3448047
              Satisfied#Mansa#Customary  |  -.2956435   .6183577    -0.48   0.633    -1.507602    .9163153
         Satisfied#Samfya#Non-compliant  |  -.1800008   .6377472    -0.28   0.778    -1.429962    1.069961
              Satisfied#Samfya#Informal  |   .8933457   .6383552     1.40   0.162    -.3578075    2.144499
             Satisfied#Samfya#Customary  |   .0644839   .6295225     0.10   0.918    -1.169358    1.298325
                                         |
-----------------------------------------+----------------------------------------------------------------
participant_id                           |
                               var(_cons)|    .757269   .0598686                      .6485678    .8841888
----------------------------------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 421.06      Prob >= chibar2 = 0.0000

Average marginal effects                                Number of obs = 14,289
Model VCE: OIM

Expression: Marginal predicted mean, predict()
dy/dx wrt:  2.tax_amount 3.tax_amount 1.start_provision 3.start_provision 2.who_to_pay 2.how_to_pay 3.how_to_pay
            4.how_to_pay 2.perks 3.perks

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
1.tax_amount |  (base outcome)
-------------+----------------------------------------------------------------
2.tax_amount |
         _at |
          1  |          .  (not estimable)
          2  |  -.0754126   .0101542    -7.43   0.000    -.0953145   -.0555107
-------------+----------------------------------------------------------------
3.tax_amount |
         _at |
          1  |          .  (not estimable)
          2  |  -.1150761   .0102856   -11.19   0.000    -.1352354   -.0949167
-------------+----------------------------------------------------------------
1.start_pr~n |
         _at |
          1  |          .  (not estimable)
          2  |  -.0048798   .0102896    -0.47   0.635     -.025047    .0152873
-------------+----------------------------------------------------------------
2.start_pr~n |  (base outcome)
-------------+----------------------------------------------------------------
3.start_pr~n |
         _at |
          1  |          .  (not estimable)
          2  |   -.025796   .0102893    -2.51   0.012    -.0459626   -.0056294
-------------+----------------------------------------------------------------
1.who_to_pay |  (base outcome)
-------------+----------------------------------------------------------------
2.who_to_pay |
         _at |
          1  |          .  (not estimable)
          2  |   -.007873   .0084198    -0.94   0.350    -.0243754    .0086294
-------------+----------------------------------------------------------------
1.how_to_pay |  (base outcome)
-------------+----------------------------------------------------------------
2.how_to_pay |
         _at |
          1  |          .  (not estimable)
          2  |  -.0092407   .0118858    -0.78   0.437    -.0325364     .014055
-------------+----------------------------------------------------------------
3.how_to_pay |
         _at |
          1  |          .  (not estimable)
          2  |  -.0068493   .0118607    -0.58   0.564    -.0300958    .0163973
-------------+----------------------------------------------------------------
4.how_to_pay |
         _at |
          1  |          .  (not estimable)
          2  |  -.0012842   .0119188    -0.11   0.914    -.0246446    .0220761
-------------+----------------------------------------------------------------
1.perks      |  (base outcome)
-------------+----------------------------------------------------------------
2.perks      |
         _at |
          1  |          .  (not estimable)
          2  |  -.0309668   .0102418    -3.02   0.002    -.0510404   -.0108931
-------------+----------------------------------------------------------------
3.perks      |
         _at |
          1  |          .  (not estimable)
          2  |  -.0236984   .0102603    -2.31   0.021    -.0438083   -.0035886
------------------------------------------------------------------------------

[Nicolas Orgeira]

I run into a comparable issue when running a different margins, where instead of the dichotomized variable=0 not being estimable, it's when the dichotomized variable=1 is not estimable. See margins output below after running the same regression, but with a different dichotomized variable

eststo m_tot_public_services2: margins tax_amount#tot_public_services2 start_provision#tot_public_services2 who_to_pay#tot_public_services2 how_to_pay#tot_public_services2 perks#tot_public_services2, post

Predictive margins                                      Number of obs = 14,289
Model VCE: OIM

Expression: Marginal predicted mean, predict()

------------------------------------------------------------------------------------------------------
                                     |            Delta-method
                                     |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------------------------------+----------------------------------------------------------------
     tax_amount#tot_public_services2 |
                           Low#<4/8  |   .7052977   .0088933    79.31   0.000     .6878672    .7227282
                          Low#>=4/8  |          .  (not estimable)
                        Medium#<4/8  |   .6060979   .0095237    63.64   0.000     .5874318     .624764
                       Medium#>=4/8  |          .  (not estimable)
                          High#<4/8  |   .5528407   .0097624    56.63   0.000     .5337069    .5719746
                         High#>=4/8  |          .  (not estimable)
                                     |
start_provision#tot_public_services2 |
               6 months before#<4/8  |    .630639    .009324    67.64   0.000     .6123643    .6489138
              6 months before#>=4/8  |          .  (not estimable)
                  At same time#<4/8  |   .6365912   .0093163    68.33   0.000     .6183316    .6548508
                 At same time#>=4/8  |          .  (not estimable)
                6 months after#<4/8  |   .5973839   .0094801    63.01   0.000     .5788032    .6159646
               6 months after#>=4/8  |          .  (not estimable)
                                     |
     who_to_pay#tot_public_services2 |
                 Local council#<4/8  |   .6251317   .0080561    77.60   0.000      .609342    .6409213
                Local council#>=4/8  |          .  (not estimable)
                         Chief#<4/8  |   .6176897   .0081133    76.13   0.000     .6017879    .6335916
                        Chief#>=4/8  |          .  (not estimable)
                                     |
     how_to_pay#tot_public_services2 |
                     In person#<4/8  |   .6263347    .010486    59.73   0.000     .6057826    .6468868
                    In person#>=4/8  |          .  (not estimable)
              Someone collects#<4/8  |   .6197929   .0105489    58.75   0.000     .5991174    .6404684
             Someone collects#>=4/8  |          .  (not estimable)
                          USSD#<4/8  |    .610482   .0104539    58.40   0.000     .5899926    .6309713
                         USSD#>=4/8  |          .  (not estimable)
                    Mobile app#<4/8  |    .629166   .0105454    59.66   0.000     .6084973    .6498347
                   Mobile app#>=4/8  |          .  (not estimable)
                                     |
          perks#tot_public_services2 |
       Participatory budgeting#<4/8  |   .6391709   .0092555    69.06   0.000     .6210303    .6573114
      Participatory budgeting#>=4/8  |          .  (not estimable)
              Submit complaint#<4/8  |   .6115988   .0094239    64.90   0.000     .5931283    .6300692
             Submit complaint#>=4/8  |          .  (not estimable)
             Vote for services#<4/8  |   .6130976   .0094505    64.87   0.000      .594575    .6316202
            Vote for services#>=4/8  |          .  (not estimable)
------------------------------------------------------------------------------------------------------

[Erik Ruzek]

You have pretty large standard errors in the first melogit model shown. It makes me wonder about how much data you have in these categorical variables you are interacting with the other variables.

[Nicolas Orgeira]

Thanks Erik Ruzek for your comments - nearly all observations (14,289/14,376) were included in the model, but the dichotomous public_service_satisf's mean value is very high (0.82), which may explains these large CI. But even when using a variable with more variations (tot_public_services2 above), I still experience the same issue estimating margins.

tab tot_public_services2, nol missing

tot_public_ |
  services2 |      Freq.     Percent        Cum.
------------+-----------------------------------
          0 |      9,444       65.69       65.69
          1 |      4,890       34.02       99.71
          . |         42        0.29      100.00
------------+-----------------------------------
      Total |     14,376      100.00

tab public_service_satisf, nol missing

     Public |
   services |
satisfactio |
          n |      Freq.     Percent        Cum.
------------+-----------------------------------
          0 |      2,436       16.94       16.94
          1 |     11,886       82.68       99.62
          . |         54        0.38      100.00
------------+-----------------------------------
      Total |     14,376      100.00

[Nicolas Orgeira]

My margins are estimable when adding noestimcheck at the end. I've read that margins may not be estimable when one of the coefficients could not be computed in the initial regression, as it was in my initial regression where one of the control variable was dropped due to collinearity. This variable was not included in the margins command though. I just wanted to check if I am proceeding correctly by just adding noestimcheck

[Erik Ruzek]

I personally would not do that. This is happening (likely) because you do not have (or have almost no) data in specific cells in the table of the crosstabs of the variables you are interacting. I would instead simplify my model so that I have adequate coverage in the cells of most interest.

[Nicolas Orgeira]

Thanks Erik Ruzek, that's a good point. I'll review my models accordingly, thanks for the suggestion