Latent Class Analysis Estat Lcmean: Too Many Options Error - r(1003)

[selim keskin]

Hello!

I have been trying to use Latent Class Analysis in Stata 15.

Here is my code:

 gsem(i.x1 i.x2 i.x3 i.x4 i.x5 <-, mlogit) (x6 x7 x8 x9 x10 x11 x12 x13 <-, logit), lclass(c 5) iterate(100)

After that I type:

estat lcprop
estat lcmean

Lcprop works fine, but Stata does not estimate estat lcmean, and it gives the following error:

estat lcmean
too many options
r(1003);

I have not specified any option, and I couldn't find anything in Stata manual. What could be the reason for this?

Note 1: I had to use iteration because after the 65ish iteration, the estimation is stuck in the same likelihood forever. Stata Manuel says that it can be a solution to this problem.
Note 2: As far as I know, in order to see the factor components, I had to specify the variables with the i prefix, and the estimation should be carried out in multinomial logit.

Thanks

[Weiwen Ng]

Note 1: I had to use iteration because after the 65ish iteration, the estimation is stuck in the same likelihood forever. Stata Manuel says that it can be a solution to this problem.

With respect, that's completely wrong. The manual said you can stop at a certain number of iterations, after which you need to examine the parameter estimates and see which ones are causing you trouble (frequently, they'll have missing standard errors). I phrased the first sentence in very strong wording not as a personal attack, but to alert you to a critical error. Without allowing the estimation algorithm to converge, you do not have maximum likelihood estimates for your parameters. Your results would be invalid. It is entirely possible that reviewers who aren't that well-versed in this subject would miss this, particularly if you didn't report standard errors. This means that you could successfully submit a paper, but several years down the road you wind up on Retraction Watch.

In my experience, latent class models with logit indicators can fail to converge if the intercept hits + or -15 - that corresponds to a probability of nearly 1 or nearly 0. Recall that the support for the logit function, takes probabilities in the range (0, 1) and transforms them to log-odds - note that I didn't type [0, 1] to mean that the range includes 0 and 1! I believe the default practice in MPlus is to constrain the intercepts (i.e. don't estimate them, treat them as certain) when this happens, and that program will flag them for the user. You can manually constrain the parameters, but it's incumbent on you to understand why you're doing that. Don't jump blindly in to this!

I can't give any insight into the "too many options" error. When I take a stock Stata dataset and artificially limit the number of iterations, estat lcmeans will still report results. I would contact Stata support, as I have a hunch this is some sort of internal error.

Also, the i. prefix is not necessary, even if you are treating the indicators as multinomial logit.

[selim keskin]

Thank you very much Weiwen, and thanks particularly for the shout out! It is important to learn one's mistake before going straight into false positives! Thanks a lot for this great comment!

[selim keskin]

Dear Weiwen and Stata Community,

After I examine the topics in which the discussions on LCA are carried out, I review my models and find that the problem is indeed certain variables trespassing the 15/-15 thresholds.

Following the suggestions, I first estimate the models with nonrtolerance, then constrained these variables, and reestimate my models without the nonrtolerance option. I also estimated the models with 50 and 100 different seeds, and found that the matrices listing the log likelihoods do not suggest any other local maxima that can hamper my results.

Yet, I have one question:
- In some of the models (especially after class option 4), after I constraine the said variables and reestimate the models, some other variables again trespass the 15/-15 thresholds. Should I iterate the same procedure in this second time (and if need be, the third or fourth time) up until the point where the usual estimation procedure of Stata (without nonrtolerance) is completed?

Thanks again!

[selim keskin]

let me bump this one time.

[Weiwen Ng]

...
Following the suggestions, I first estimate the models with nonrtolerance, then constrained these variables, and reestimate my models without the nonrtolerance option. I also estimated the models with 50 and 100 different seeds, and found that the matrices listing the log likelihoods do not suggest any other local maxima that can hamper my results.

Yet, I have one question:
- In some of the models (especially after class option 4), after I constraine the said variables and reestimate the models, some other variables again trespass the 15/-15 thresholds. Should I iterate the same procedure in this second time (and if need be, the third or fourth time) up until the point where the usual estimation procedure of Stata (without nonrtolerance) is completed?

Thanks again!

I'm answering this based only on opinion. I don't know that there is solid guidance for this. I would constrain all the variables that surpass those thresholds.

I asserted earlier that MPlus did these constraints automatically. I didn't offer evidence for that assertion at the time. Here is one thread on MPlus' discussion forum where Linda Muthen responds (NB: I believe she programmed MPlus, and she and Bengt Muthen do a bunch of theoretical work in the LCA field). She also said that

Having thresholds fixed at -15 or +15 is not a problem. It just means that for endorsing that particular item the probability is one or zero in that class. It can be helpful for defining the class.

This was in response to a query about if you can generally trust final results that contain these constraints or not.

[selim keskin]

Thanks a lot Weiwen! I really appreciate all your comments here and in the other threads. They are extremely helpful!

[Trenton Mize]

I ran into this problem as well so will share the solution here in case it proves useful for others. As far as I can tell, estat lcmean is just a convenience wrapper for margins. The "too many options" error occurs when the number of items X number of classes combinations produces more needed predictions than margins allows. So the solution is just to use margins but ask for fewer things at once (e.g. one margins command per class). E.g.

. use "https://www.stata-press.com/data/r17/gsem_lca1", clear
(Latent class analysis)

. 
. gsem (accident play insurance stock <- ), logit lclass(C 2)

Fitting class model:

Iteration 0:   (class) log likelihood = -149.71979  
Iteration 1:   (class) log likelihood = -149.71979  

Fitting outcome model:

Iteration 0:   (outcome) log likelihood = -403.97142  
Iteration 1:   (outcome) log likelihood = -398.15909  
Iteration 2:   (outcome) log likelihood = -397.81953  
Iteration 3:   (outcome) log likelihood =  -397.8164  
Iteration 4:   (outcome) log likelihood =  -397.8164  

Refining starting values:

Iteration 0:   (EM) log likelihood = -570.24204
Iteration 1:   (EM) log likelihood = -576.20485
Iteration 2:   (EM) log likelihood = -577.41464
Iteration 3:   (EM) log likelihood = -576.88554
Iteration 4:   (EM) log likelihood = -575.59242
Iteration 5:   (EM) log likelihood = -573.90567
Iteration 6:   (EM) log likelihood = -571.99868
Iteration 7:   (EM) log likelihood = -569.97482
Iteration 8:   (EM) log likelihood = -567.90955
Iteration 9:   (EM) log likelihood = -565.86392
Iteration 10:  (EM) log likelihood = -563.88815
Iteration 11:  (EM) log likelihood = -562.02165
Iteration 12:  (EM) log likelihood = -560.29231
Iteration 13:  (EM) log likelihood = -558.71641
Iteration 14:  (EM) log likelihood = -557.29974
Iteration 15:  (EM) log likelihood = -556.03949
Iteration 16:  (EM) log likelihood = -554.92679
Iteration 17:  (EM) log likelihood = -553.94914
Iteration 18:  (EM) log likelihood = -553.09241
Iteration 19:  (EM) log likelihood = -552.34233
Iteration 20:  (EM) log likelihood = -551.68539
note: EM algorithm reached maximum iterations.

Fitting full model:

Iteration 0:   log likelihood = -504.62913  
Iteration 1:   log likelihood = -504.47255  
Iteration 2:   log likelihood = -504.46773  
Iteration 3:   log likelihood = -504.46767  
Iteration 4:   log likelihood = -504.46767  

Generalized structural equation model                      Number of obs = 216
Log likelihood = -504.46767

------------------------------------------------------------------------------
             | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
1.C          |  (base outcome)
-------------+----------------------------------------------------------------
2.C          |
       _cons |     -0.948      0.289   -3.285   0.001       -1.514      -0.382
------------------------------------------------------------------------------

Class:    1        

Response: accident 
Family:   Bernoulli
Link:     Logit    

Response: play     
Family:   Bernoulli
Link:     Logit    

Response: insurance
Family:   Bernoulli
Link:     Logit    

Response: stock    
Family:   Bernoulli
Link:     Logit    

------------------------------------------------------------------------------
             | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
accident     |
       _cons |      0.913      0.197    4.623   0.000        0.526       1.300
-------------+----------------------------------------------------------------
play         |
       _cons |     -0.710      0.225   -3.156   0.002       -1.151      -0.269
-------------+----------------------------------------------------------------
insurance    |
       _cons |     -0.601      0.212   -2.833   0.005       -1.018      -0.185
-------------+----------------------------------------------------------------
stock        |
       _cons |     -1.880      0.334   -5.633   0.000       -2.534      -1.226
------------------------------------------------------------------------------

Class:    2        

Response: accident 
Family:   Bernoulli
Link:     Logit    

Response: play     
Family:   Bernoulli
Link:     Logit    

Response: insurance
Family:   Bernoulli
Link:     Logit    

Response: stock    
Family:   Bernoulli
Link:     Logit    

------------------------------------------------------------------------------
             | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
accident     |
       _cons |      4.983      3.746    1.330   0.183       -2.359      12.325
-------------+----------------------------------------------------------------
play         |
       _cons |      2.747      1.166    2.357   0.018        0.462       5.032
-------------+----------------------------------------------------------------
insurance    |
       _cons |      2.535      0.964    2.628   0.009        0.644       4.425
-------------+----------------------------------------------------------------
stock        |
       _cons |      1.203      0.536    2.244   0.025        0.153       2.254
------------------------------------------------------------------------------

. 
. estat lcmean

Latent class marginal means                                Number of obs = 216

--------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.     [95% conf. interval]
-------------+------------------------------------------------
1            |
    accident |      0.714      0.040         0.629       0.786
        play |      0.330      0.050         0.240       0.433
   insurance |      0.354      0.049         0.266       0.454
       stock |      0.132      0.038         0.073       0.227
-------------+------------------------------------------------
2            |
    accident |      0.993      0.025         0.086       1.000
        play |      0.940      0.066         0.614       0.994
   insurance |      0.927      0.066         0.656       0.988
       stock |      0.769      0.095         0.538       0.905
--------------------------------------------------------------

. 
. margins,        predict(outcome(accident) class(1)) ///
>                         predict(outcome(play) class(1)) ///
>                         predict(outcome(insurance) class(1)) ///
>                         predict(outcome(stock) class(1))
warning: prediction constant over observations.
warning: prediction constant over observations.
warning: prediction constant over observations.
warning: prediction constant over observations.

Predictive margins                                         Number of obs = 216
Model VCE: OIM

1._predict: Predicted mean (Would testify against friend in accident case in class 1.C),
            predict(outcome(accident) class(1))
2._predict: Predicted mean (Would give negative review of friend's play in class 1.C),
            predict(outcome(play) class(1))
3._predict: Predicted mean (Would disclose health concerns to friend's insurance company in ,
            predict(outcome(insurance) class(1))
4._predict: Predicted mean (Would keep company secret from friend in class 1.C),
            predict(outcome(stock) class(1))

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
    _predict |
          1  |      0.714      0.040   17.681   0.000        0.634       0.793
          2  |      0.330      0.050    6.632   0.000        0.232       0.427
          3  |      0.354      0.049    7.291   0.000        0.259       0.449
          4  |      0.132      0.038    3.453   0.001        0.057       0.208
------------------------------------------------------------------------------

.                         
. margins,        predict(outcome(accident) class(2)) ///
>                         predict(outcome(play) class(2)) ///
>                         predict(outcome(insurance) class(2)) ///
>                         predict(outcome(stock) class(2))
warning: prediction constant over observations.
warning: prediction constant over observations.
warning: prediction constant over observations.
warning: prediction constant over observations.

Predictive margins                                         Number of obs = 216
Model VCE: OIM

1._predict: Predicted mean (Would testify against friend in accident case in class 2.C),
            predict(outcome(accident) class(2))
2._predict: Predicted mean (Would give negative review of friend's play in class 2.C),
            predict(outcome(play) class(2))
3._predict: Predicted mean (Would disclose health concerns to friend's insurance company in ,
            predict(outcome(insurance) class(2))
4._predict: Predicted mean (Would keep company secret from friend in class 2.C),
            predict(outcome(stock) class(2))

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
    _predict |
          1  |      0.993      0.025   39.219   0.000        0.944       1.043
          2  |      0.940      0.066   14.240   0.000        0.810       1.069
          3  |      0.927      0.066   14.112   0.000        0.798       1.055
          4  |      0.769      0.095    8.079   0.000        0.583       0.956
------------------------------------------------------------------------------