Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • When i run the melogit command I get error messge ":cannot compute an improvement -- discontinuous region encountered "

    I am trying to compute the ICC model for my data but the model will not converge when I use the melogit command

    Background on data
    outcome is the dependent variable with 2 levels 1 and 0.
    0 makes up 90% of the data and 1 makes up 10% of the data
    group is the grouping variable



    The command and the output

    Code:
    melogit outcome || group:
    Code:
    Fitting fixed-effects model:
    
    Iteration 0:   log likelihood = -20457.768  
    Iteration 1:   log likelihood =  -7770.252  
    Iteration 2:   log likelihood = -7735.1396  
    Iteration 3:   log likelihood = -7734.4482  
    Iteration 4:   log likelihood = -7734.4481  
    
    Refining starting values:
    
    Grid node 0:   log likelihood = -5884.3781
    
    Fitting full model:
    
    Iteration 0:   log likelihood = -5884.3781  
    Iteration 1:   log likelihood = -5671.9642  
    Iteration 2:   log likelihood = -5538.9426  
    Iteration 3:   log likelihood = -5521.4719  
    Iteration 4:   log likelihood =  -5508.246  
    Iteration 5:   log likelihood = -5498.0299  
    Iteration 6:   log likelihood =  -5490.132  
    Iteration 7:   log likelihood = -5484.0462  
    Iteration 8:   log likelihood = -5479.3835  
    Iteration 9:   log likelihood = -5475.8658  
    Iteration 10:  log likelihood = -5474.4419  
    cannot compute an improvement -- discontinuous region encountered
    r(430);

    thank you for your response.
    Last edited by Sohe Aja; 05 Mar 2021, 13:30.

  • #2
    The output doesn't, in this case, provide any clues. I have a couple of suggestions:

    1. While a 10% vs 90% distribution is not, per se, a problem, if the sample is small, the small number of 1's might be. What does -tab outcome- show?
    2. It could also be a problem if the 1 values of outcome are all bottled up in just a small number of groups, the of the groups being all zeroes.
    3. What does the distribution of group sizes look like? If you have a lot of singleton groups, or many very small groups, that might be contributing.
    Code:
    by group, sort: gen group_size = _N if _n == 1
    summ group_size, detail
    Finally, you have referred to this as an ICC model, and I understand your meaning. But the intraclass correlations derived from logistic models are usually not very useful. I'm wondering what you have in mind to do with the results: for most purposes, even with a dichotomous outcome, the ICC you want is usually the one that comes out of -mixed-. (For example, if you need an ICC to calculate a design effect to plan some future study, the one from -melogit- is not suitable for that purpose, the one from -mixed- is.)

    Comment


    • #3
      Thank you, Clyde.

      Response to your questions.

      1.

      Code:
                 l |      Freq.     Percent        Cum.
      -------------+-----------------------------------
                0 |    199,156       99.36       99.36
                1 |      1,278        0.64      100.00
      -------------+-----------------------------------
             Total |    200,434      100.00
      I just noticed the groups with the 1s is less than 10%

      2. Yes. they are bottled in a small number of groups.
      3. I have a number of small groups.




      when I run the command

      Code:
       by group, sort: gen group_size = _N if _n == 1 summ group_size, detail

      result


      Code:
                             group_size
      -------------------------------------------------------------
            Percentiles      Smallest
       1%            1              1
       5%           19              1
      10%           59              1       Obs                 383
      25%          132              1       Sum of Wgt.         383
      
      50%          351                      Mean           523.3264
                              Largest       Std. Dev.      511.8457
      75%          718           2040
      90%         1340           2097       Variance         261986
      95%         1613           2149       Skewness       1.292729
      99%         2040           2379       Kurtosis        3.95972

      I hope this helps describe the issue.
      Last edited by Sohe Aja; 05 Mar 2021, 15:02.

      Comment


      • #4
        Well, although the percentage of 1's is very low, given the large N, the total number of 1's is over 1,000--so that should not be a problem.

        It may well be the small groups, although it looks like you only have four that are actually singletons. If there is some real-world meaningful way to combine them into larger groups, or absorb them into existing larger groups. Perhaps some of the groups are geographically near each other, or have some other similarity that makes this defensible.

        Another approach is to try the -difficult- option in your -melogit- command. Sometimes that will get a non-convergent model to converge. You might also try specifying a different integration method, such as -int(mcaghermite)- or -intmethod(pcaghermite)- or -intmethod(ghermite)-.

        In any case, given that you have no continuous predictors in the model, if nothing else you can speed up the processing (so failing attempts won't take so long to reach their dismal conclusion) by doing this:

        Code:
        collapse (sum) outcome (count) denom = outcome, by(group)
        melogit outcome, binomial(denom) || group:
        (By the way, I have occasionally seen this trick of collapsing the data set into group-level observations cause a non-convergent model to converge. So give it a try. I have no idea why it would have that effect--in principle this approach rearranges the data without changing anything about the calculation of the likelihood, but sometimes, inexplicably, it does work.)

        Comment


        • #5
          Thank you, Clyde. I will try the -difficult- option first, and if that does not work, I will absorb the smaller groups into larger ones, and then I will try the code and report back on my findings.

          Comment


          • #6
            Originally posted by Clyde Schechter View Post
            Well, although the percentage of 1's is very low, given the large N, the total number of 1's is over 1,000--so that should not be a problem.

            It may well be the small groups, although it looks like you only have four that are actually singletons. If there is some real-world meaningful way to combine them into larger groups, or absorb them into existing larger groups. Perhaps some of the groups are geographically near each other, or have some other similarity that makes this defensible.

            Another approach is to try the -difficult- option in your -melogit- command. Sometimes that will get a non-convergent model to converge. You might also try specifying a different integration method, such as -int(mcaghermite)- or -intmethod(pcaghermite)- or -intmethod(ghermite)-.

            In any case, given that you have no continuous predictors in the model, if nothing else you can speed up the processing (so failing attempts won't take so long to reach their dismal conclusion) by doing this:

            Code:
            collapse (sum) outcome (count) denom = outcome, by(group)
            melogit outcome, binomial(denom) || group:
            (By the way, I have occasionally seen this trick of collapsing the data set into group-level observations cause a non-convergent model to converge. So give it a try. I have no idea why it would have that effect--in principle this approach rearranges the data without changing anything about the calculation of the likelihood, but sometimes, inexplicably, it does work.)
            It worked! Thank you!
            I took the "If there is some real-world meaningful way to combine them into larger groups or absorb them into existing larger groups" route but instead of combining them. I selected the treatment groups instead of the entire study population. This helped reduce the disproportionality between the levels (0s,1s) in my dependent variable and also the number of zeros in my grouping variable.

            Comment


            • #7
              Originally posted by Clyde Schechter View Post
              The output doesn't, in this case, provide any clues. I have a couple of suggestions:



              Finally, you have referred to this as an ICC model, and I understand your meaning. But the intraclass correlations derived from logistic models are usually not very useful. I'm wondering what you have in mind to do with the results: for most purposes, even with a dichotomous outcome, the ICC you want is usually the one that comes out of -mixed-. (For example, if you need an ICC to calculate a design effect to plan some future study, the one from -melogit- is not suitable for that purpose, the one from -mixed- is.)

              I intend using the ICC model to determine if there is enough variance in level 2 to account for the use of the HGLM. Also, i want to use it as a base model in comparism to when i include the level 1 variables and the level 2 variables.

              Comment


              • #8
                Originally posted by Clyde Schechter View Post
                But the intraclass correlations derived from logistic models are usually not very useful. . . . for most purposes, even with a dichotomous outcome, the ICC you want is usually the one that comes out of -mixed-. (For example, if you need an ICC to calculate a design effect to plan some future study, the one from -melogit- is not suitable for that purpose, the one from -mixed- is.)
                Clyde, could you elaborate on that? I hadn't been aware of it.

                Comment


                • #9
                  Actually, I'm going to retract that. It may be correct, but reviewing where I got that from, I'm not sure.

                  This arose about 8 years ago when I was trying to plan a study of an intervention to reduce the frequency of an adverse event from about 0.5 per 1,000 opportunities to 0.25 per 1,000. The nature of the data was inherently longitudinal and there were random effects in a multiple membership model, and it also exhibited a very high intraclass correlation. And we were constrained to a fixed number of entities at one crossed level because that's just how many there were. The sample size calculations used some software that a colleague had developed for the purpose; it was not commercially available. We found that using the ICC obtained from a previous mixed-effects logistic model of a very similar study led to concluding that we need a sample size of several billion entities at one level (Each cross with the other level containing around 100 observations), which clearly was not going to happen and intuitively made no sense given prior successful experience with a much smaller sample size (a few million). But when we used the intraclass correlation from a mixed effects linear model of the same data, it produced a more plausible sample size, and simulations confirmed that that was at least approximately correct. So that was the experience I based that remark on.

                  But thinking about it more carefully, I do not have a deep understanding of sample size calculations for mixed effects logistic models, especially when the random effects are multiple-membership instead of nested, and the sample size software was a black-box to me. Another plausible explanation is that the sample size software was not robust to the somewhat extreme parameters we were using it with and simply broke down. I don't know.

                  (Suffice it to say that we did subsequently do the study we were planning with the sample size in the millions, and it worked out fine.)
                  Last edited by Clyde Schechter; 09 Mar 2021, 11:31.

                  Comment


                  • #10
                    Clyde, thank you. I appreciate your getting back on it. I've no idea either what could have been going on with the software, but your explanation makes sense.

                    Comment

                    Working...
                    X