xtprobit stuck at rho estimation

Ariel Karlinsky

Join Date: Jun 2015

Posts: 491
#1

xtprobit stuck at rho estimation

28 Sep 2016, 04:42

ML models will sometimes get stuck at "backing up" - nothing new about that.
The difficulty I'm currently having is that the model "gets stuck" at estimating the rho of the model. something which I also cannot control and inspect with the "iterate" option.
This does not happen when I drop one of the factor variables which has many levels (~250). Of course I continue in exploring the model with this variable or several other transformations of it (it's a geographic control variable, so I'm attempting larger geographic areas for example). I would still like to know how I can "control" the maximization problem at this particular stage, to perhaps find out what's wrong. Stata gives no indication that something is wrong, but the last estimation was stuck after giving the log likelihood of rho = 0.4 for almost 14 hours...

Stata version is 14.2 SE

Stata output illustration (this particular model on this dataset of course estimates without issue, it's from the help file):

Code:

*Time is 00:00* . webuse union (NLS Women 14-24 in 1968) . xtprobit union age grade i.not_smsa south##c.year Fitting comparison model: Iteration 0: log likelihood = -13864.23 Iteration 1: log likelihood = -13545.541 Iteration 2: log likelihood = -13544.385 Iteration 3: log likelihood = -13544.385 Fitting full model: rho = 0.0 log likelihood = -13544.385 rho = 0.1 log likelihood = -12237.655 rho = 0.2 log likelihood = -11590.282 rho = 0.3 log likelihood = -11211.185 rho = 0.4 log likelihood = -10981.319 --Break-- *Time is 14:00*
Tags: None

Alfonso Sánchez-Peñalver

Join Date: Mar 2014
Posts: 432

28 Sep 2016, 07:24

Not sure why it's giving you trouble. Have you tried estimating it with meprobit?

Here the xtprobit estimation:

Code:

. webuse union
(NLS Women 14-24 in 1968)

. xtprobit union age grade i.not_smsa south##c.year, nolog

Random-effects probit regression                Number of obs      =     26200
Group variable: idcode                          Number of groups   =      4434

Random effects u_i ~ Gaussian                   Obs per group: min =         1
                                                               avg =       5.9
                                                               max =        12

Integration method: mvaghermite                 Integration points =        12

                                                Wald chi2(6)       =    220.91
Log likelihood  = -10552.225                    Prob > chi2        =    0.0000

------------------------------------------------------------------------------
       union |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .0082967   .0084599     0.98   0.327    -.0082843    .0248778
       grade |   .0482731   .0099469     4.85   0.000     .0287776    .0677686
  1.not_smsa |   -.139657   .0460548    -3.03   0.002    -.2299227   -.0493913
     1.south |  -1.584394    .358473    -4.42   0.000    -2.286989   -.8818002
        year |  -.0039854   .0088399    -0.45   0.652    -.0213113    .0133406
             |
south#c.year |
          1  |   .0134017   .0044622     3.00   0.003     .0046559    .0221475
             |
       _cons |  -1.668202   .4751819    -3.51   0.000    -2.599542   -.7368628
-------------+----------------------------------------------------------------
    /lnsig2u |   .6103616   .0458783                      .5204418    .7002814
-------------+----------------------------------------------------------------
     sigma_u |    1.35687   .0311255                      1.297217    1.419267
         rho |   .6480233   .0104643                      .6272511    .6682502
------------------------------------------------------------------------------
Likelihood-ratio test of rho=0: chibar2(01) =  5984.32 Prob >= chibar2 = 0.000

And here the equivalent meprobit estimation:

Code:

. meprobit union age grade i.not_smsa south##c.year || idcode:, intpoints(12) nolog

Mixed-effects probit regression                 Number of obs      =     26200
Group variable:          idcode                 Number of groups   =      4434

                                                Obs per group: min =         1
                                                               avg =       5.9
                                                               max =        12

Integration method: mvaghermite                 Integration points =        12

                                                Wald chi2(6)       =    220.68
Log likelihood = -10552.225                     Prob > chi2        =    0.0000
------------------------------------------------------------------------------
       union |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .0082967   .0084647     0.98   0.327    -.0082938    .0248871
       grade |   .0482905   .0099525     4.85   0.000      .028784     .067797
  1.not_smsa |  -.1396376   .0460689    -3.03   0.002    -.2299309   -.0493443
     1.south |  -1.584809   .3585267    -4.42   0.000    -2.287509   -.8821098
        year |  -.0039848   .0088447    -0.45   0.652      -.02132    .0133505
             |
south#c.year |
          1  |   .0134056   .0044628     3.00   0.003     .0046587    .0221525
             |
       _cons |  -1.668914   .4754444    -3.51   0.000    -2.600768   -.7370606
-------------+----------------------------------------------------------------
idcode       |
   var(_cons)|   1.843666    .085112                      1.684174    2.018262
------------------------------------------------------------------------------
LR test vs. probit regression:   chibar2(01) =  5984.32 Prob>=chibar2 = 0.0000

Alfonso Sanchez-Penalver

Comment

Ariel Karlinsky

Join Date: Jun 2015
Posts: 491

01 Oct 2016, 03:29

Thanks for the suggestion! trying meprobit yields the following errors:

Code:

 cap noi eststo: meprobit `y' `x' || id: , intpoints(12)

Fitting fixed-effects model:

Iteration 0:   log likelihood = -817836.83  
Iteration 1:   log likelihood = -817374.06  
Iteration 2:   log likelihood = -817374.01  
Iteration 3:   log likelihood = -817374.01  

Refining starting values:

Grid node 0:   log likelihood = -780638.83

Fitting full model:

                     J():  3900  unable to allocate real <tmp>[152725,38503]
 _gsem_eval_chol__work():     -  function returned error
       _gsem_eval_chol():     -  function returned error
      mopt__calluser_v():     -  function returned error
       opt__eval_nr_v2():     -  function returned error
             opt__eval():     -  function returned error
opt__looputil_iter0_common():     -  function returned error
opt__looputil_iter0_nr():     -  function returned error
          opt__loop_nr():     -  function returned error
            _moptimize():     -  function returned error
           Mopt_maxmin():     -  function returned error
                 <istmt>:     -  function returned error

r; t=915.38 9:45:13

Comment

Stephen Jenkins

Join Date: Apr 2014
Posts: 1433

01 Oct 2016, 04:04

Ariel: Are you fully updated? Whatever, I suspect that it's time for you to contact Technical Support. The issues you relate may be related to your own installation, I suspect. I have no problem running the code Alfonso tried. [If you post further output, please ensure you show the definition of any relevant local macros -- we can't see what your "`y'" and "`x'" are in your last post.]

Code:

. about

Stata/MP 14.2 for Windows (64-bit x86-64)
Revision 29 Sep 2016
Copyright 1985-2015 StataCorp LP

Total physical memory:    16730612 KB
Available physical memory: 13081804 KB

<snip>

. webuse union
(NLS Women 14-24 in 1968)

. meprobit union age grade i.not_smsa south##c.year || idcode:, intpoints(12)

Fitting fixed-effects model:

Iteration 0:   log likelihood = -13560.682 
Iteration 1:   log likelihood = -13544.386 
Iteration 2:   log likelihood = -13544.385 

Refining starting values:

Grid node 0:   log likelihood = -10852.765

Fitting full model:

Iteration 0:   log likelihood = -10852.765 
Iteration 1:   log likelihood = -10585.248 
Iteration 2:   log likelihood = -10552.548 
Iteration 3:   log likelihood = -10552.226 
Iteration 4:   log likelihood = -10552.225 

Mixed-effects probit regression                 Number of obs     =     26,200
Group variable:          idcode                 Number of groups  =      4,434

                                                Obs per group:
                                                              min =          1
                                                              avg =        5.9
                                                              max =         12

Integration method: mvaghermite                 Integration pts.  =         12

                                                Wald chi2(6)      =     220.68
Log likelihood = -10552.225                     Prob > chi2       =     0.0000
------------------------------------------------------------------------------
       union |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .0082967   .0084647     0.98   0.327    -.0082938    .0248871
       grade |   .0482905   .0099525     4.85   0.000      .028784     .067797
  1.not_smsa |  -.1396376   .0460689    -3.03   0.002    -.2299309   -.0493443
     1.south |  -1.584809   .3585267    -4.42   0.000    -2.287509   -.8821098
        year |  -.0039848   .0088447    -0.45   0.652      -.02132    .0133505
             |
south#c.year |
          1  |   .0134056   .0044628     3.00   0.003     .0046587    .0221525
             |
       _cons |  -1.668914   .4754444    -3.51   0.000    -2.600768   -.7370606
-------------+----------------------------------------------------------------
idcode       |
   var(_cons)|   1.843666    .085112                      1.684174    2.018262
------------------------------------------------------------------------------
LR test vs. probit model: chibar2(01) = 5984.32       Prob >= chibar2 = 0.0000

Comment

Ariel Karlinsky

Join Date: Jun 2015

Posts: 491
#5

01 Oct 2016, 05:01

Stata is fully updated (14.2, September 29). I cannot show the values for the macros as the data is un-public and on a stand alone station.
the union example dataset was used just as an example. I'll try to contact TS
Comment

Announcement