Meologit

[Laura Andreea]

Question:

Do you know what might be the problem, and why stata does not recognizes apples as 10 groups and instead sees them as 6080 different observations?

Thank you in advance!
Andreea

Background:
We have our data in long format where:

• we have 10 repeated observations per participant.
• In total we have 608 participants (6080 rows)
• PL: purchase likelihood (dependent variable (per apple))
• Ia: implicit attitude score (per participant) (explanatory var.)
• Ea: explicit attitude score (per apple) (explanatory var.)

Data for one participant.

id	PL	ia	ea	apple
85	1	1,29	1,33	10
85	1	1,29	1,00	6
85	1	1,29	1,17	7
85	7	1,29	9,83	1
85	1	1,29	1,00	8
85	7	1,29	10,33	2
85	7	1,29	9,17	3
85	1	1,29	1,00	9
85	5	1,29	7,17	5
85	7	1,29	10,67	4

We want to model purchase likelihood where ea and ia are fixed and we have random intercepts by id and apple.

Our code is stata looks like this:
meologit PL ea ia ea##ia || id: || apple:

The way our dataset looks like, when it comes to the number of groups, we should have id 608 and apple 10. However stata dose not recognizes that.

Stata output


Mixed-effects ologit regression Number of obs = 6,080

-------------------------------------------------------------
| No. of Observations per Group
Group Variable | Groups Minimum Average Maximum
----------------+--------------------------------------------
iid | 608 10 10.0 10
AppleNR | 6,080 1 1.0 1
-------------------------------------------------------------

Integration method: mvaghermite Integration pts. = 7

Wald chi2(2) = 3934.89
Log likelihood = -6690.0858 Prob > chi2 = 0.0000
------------------------------------------------------------------------------
Purch1_r1 | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
iat | -.3780785 .1062244 -3.56 0.000 -.5862744 -.1698826
ea | 1.345597 .0214517 62.73 0.000 1.303553 1.387642
-------------+----------------------------------------------------------------
/cut1 | 4.069106 .1396939 3.795311 4.342901
/cut2 | 5.633302 .1507736 5.337791 5.928813
/cut3 | 7.143381 .1655579 6.818893 7.467868
/cut4 | 8.641514 .1813477 8.286079 8.996949
/cut5 | 10.34706 .1998057 9.955445 10.73867
/cut6 | 12.10224 .2171978 11.67654 12.52794
-------------+----------------------------------------------------------------
iid |
var(_cons)| 1.09431 .1017395 .9120174 1.31304
-------------+----------------------------------------------------------------
iid>AppleNR |
var(_cons)| 4.45e-34 2.82e-18 . .
------------------------------------------------------------------------------
LR test vs. ologit model: chibar2(01) = 507.78 Prob >= chibar2 = 0.0000

[Weiwen Ng]

Easy. You reversed the syntax for specifying the levels of the random effects. I've made that mistake before.

Also, ## will get you both main effects and interactions, so you have some redundant covariates.

meologit PL ea##ia || apple:  || id:

[Laura Andreea]

Thank you for your answer.

By reversing then I have 10 apple groups, but then ID is 6 080 and not 608. We have 608 individuals and 10 apple groups. I need stata to recognize that there are 608 individuals and 10 apples.

meologit PL ea##ia || apple: || id:


Mixed-effects ologit regression Number of obs = 6,080

-------------------------------------------------------------
| No. of Observations per Group
Group Variable | Groups Minimum Average Maximum
----------------+--------------------------------------------
apple | 10 608 608.0 608
id | 6,080 1 1.0 1
-------------------------------------------------------------

Integration method: mvaghermite Integration pts. = 7

Wald chi2(3) = 3634.37
Log likelihood = -6914.3559 Prob > chi2 = 0.0000
------------------------------------------------------------------------------
PL | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ea | 1.075282 .0264532 40.65 0.000 1.023435 1.12713
ia | -.3246146 .1357099 -2.39 0.017 -.590601 -.0586281
|
c.ea#c.ia | .0063811 .0206138 0.31 0.757 -.0340212 .0467834
-------------+----------------------------------------------------------------
/cut1 | 3.015399 .1819715 2.658742 3.372057
/cut2 | 4.382017 .1867818 4.015932 4.748103
/cut3 | 5.681451 .1936236 5.301956 6.060946
/cut4 | 6.970726 .201618 6.575562 7.36589
/cut5 | 8.451205 .2116631 8.036353 8.866057
/cut6 | 9.975174 .2216763 9.540697 10.40965
-------------+----------------------------------------------------------------
apple |
var(_cons)| .1028991 .0525075 .0378494 .2797462
-------------+----------------------------------------------------------------
apple>id |
var(_cons)| 2.99e-34 2.67e-18 . .
------------------------------------------------------------------------------
LR test vs. ologit model: chibar2(01) = 60.19 Prob >= chibar2 = 0.0000

.

[Weiwen Ng]

Side note before we proceed further: please use code delimiters to show your code and results. They format better, and the results can be more easily deciphered.

I should have inspected your data example. It looks like you gave descriptions of 10 different apples to 608 participants (or actual apples), then asked them to rate each one. If this is correct, then you actually have a crossed effects model, not a nested model. My bad for not looking more closely. I believe your syntax should have been:

meologit PL ea##ia || _all: R.apple || id:

A hint: it looks like you manually typed that in a table. It would be easier on your end if you extracted that via the -dataex- command and put that in code delimiters. If someone wanted to import that to Stata, they could just have copied the resulting data from your post and pasted it into a command window.

dataex id PL ia ea apple if id==85

[Laura Andreea]

Thank you.

I run the new command but it cannot fit the data.

meologit PL c.ea##c.ia || _all: R.apple || id:

Fitting fixed-effects model:

Iteration 0: log likelihood = -11355.049
Iteration 1: log likelihood = -7518.2064
Iteration 2: log likelihood = -6975.6101
Iteration 3: log likelihood = -6944.5048
Iteration 4: log likelihood = -6944.4532
Iteration 5: log likelihood = -6944.4532

Refining starting values:

Grid node 0: log likelihood = -6734.3524

Fitting full model:

initial values not feasible
r(1400);

Here is the data.

* Example generated by -dataex-. To install: ssc install dataex
clear
input int id byte PL double(ia ea) byte apple
85 7 1.2936068 9.833333  1
85 7 1.2936068 10.33333  2
85 7 1.2936068 9.166667  3
85 7 1.2936068 10.66667  4
85 5 1.2936068 7.166667  5
85 1 1.2936068        1  6
85 1 1.2936068 1.166667  7
85 1 1.2936068        1  8
85 1 1.2936068        1  9
85 1 1.2936068 1.333333 10
end

------------------ copy up to and including the previous line ------------------

Listed 10 out of 6080 observations

[Laura Andreea]

meologit PL c.ea##c.ia || _all: R.apple || id:

[Weiwen Ng]

Thank you.

I run the new command but it cannot fit the data.

meologit PL c.ea##c.ia || _all: R.apple || id:

Fitting fixed-effects model:

Iteration 0: log likelihood = -11355.049
Iteration 1: log likelihood = -7518.2064
Iteration 2: log likelihood = -6975.6101
Iteration 3: log likelihood = -6944.5048
Iteration 4: log likelihood = -6944.4532
Iteration 5: log likelihood = -6944.4532

Refining starting values:

Grid node 0: log likelihood = -6734.3524

Fitting full model:

initial values not feasible
r(1400);

...

Multilevel models frequently have convergence issues like this. Crossed effects models are, I believe, even harder to fit than nested ones. This thread discusses some potential solutions. Basically, you have a 3-level model. Stata fit a model without any random effects to get starting values, and then declared that they were infeasible.

I'd start by fitting a 2-level model, saving the parameter estimates, then trying your 3-level model and asking Stata to start from the saved parameter estimates:

meologit PL c.ea##c.ia || id:
matrix b = e(b)
meologit PL c.ea##c.ia || _all: R.apple || id:, from(b)

If Stata still chokes, then I'd assume it's having trouble estimating the variance component for apple. You could try Joseph Conevey's suggestion in post #6 of the thread I linked.

Also, do verify substantively that you need a crossed effects model. In particular, did you actually give 10 identical apples (or simulacra of apples) to each participant and ask them to rate each one?

[Laura Andreea]

Hi again,

Thank you. It is getting better, but I am not there yet. Yes, I realized now that I was using the wrong model. As you said it is a crossed effects model. But the stata I am using does not have the capacity to run with all the observations I have.

[P]     error . . . . . . . . . . . . . . . . . . . . . . . . Return code 1400
        numerical overflow
        You have attempted something that, in the midst of the
        necessary calculations, has resulted in something too large
        for Stata to deal with accurately.  Most commonly, this is
        an attempt to estimate a model with too many effective
        observations.  This effective number could be reached with far
        fewer observations if you were running a frequency-weighted
        model.

Just to try I run instead with 152 observations and then it works.

meologit PL c.ea##c.ia || _all: R.apple || id:

Mixed-effects ologit regression                 Number of obs     =      1,520

-------------------------------------------------------------
                |     No. of       Observations per Group
 Group Variable |     Groups    Minimum    Average    Maximum
----------------+--------------------------------------------
           _all |          1      1,520    1,520.0      1,520
             id |        152         10       10.0         10
-------------------------------------------------------------

Integration method:     laplace

                                                Wald chi2(3)      =     575.56
Log likelihood = -1527.4565                     Prob > chi2       =     0.0000
------------------------------------------------------------------------------
          PL |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
          ea |   1.488538   .1175305    12.67   0.000     1.258183    1.718894
          ia |  -.3185732   .5388996    -0.59   0.554    -1.374797    .7376506
             |
   c.ea#c.ia |  -.0076113   .0543968    -0.14   0.889    -.1142271    .0990045
-------------+----------------------------------------------------------------
       /cut1 |   4.647436   .8259851                      3.028534    6.266337
       /cut2 |   6.417973   .8873103                      4.678876    8.157069
       /cut3 |    8.07016   .9506569                      6.206907    9.933414
       /cut4 |   9.560255   1.005031                       7.59043    11.53008
       /cut5 |   11.32699   1.073115                      9.223722    13.43025
       /cut6 |   13.42533   1.141993                      11.18707     15.6636
-------------+----------------------------------------------------------------
_all>apple   |
   var(_cons)|   .0543493   .0441047                      .0110775    .2666539
-------------+----------------------------------------------------------------
id           |
   var(_cons)|   .9808673   .2330167                      .6157397    1.562512
------------------------------------------------------------------------------
LR test vs. ologit model: chi2(2) = 103.16                Prob > chi2 = 0.0000

[Joseph Coveney]

I don't know what Stata you are using, but I suspect that numerical overflow isn't because of it.

It's not clear from

Ea: explicit attitude score (per apple) (explanatory var.)

how an apple possesses an explicit attitude as a quantitative attribute, but I assume that the value for each apple's ea is identical across participants and similarly for each participant's ia across apples. It's possible that the numerical difficulty arises in teasing apart the (orthogonal) fixed and random effects, especially with apples, where there are only ten of them. (Note the magnitude of the variance estimate for apple in the model that you were able to fit.) You might want to drop the random effect for apple and just consider it fixed only. I kind of doubt that you're going to be able to build much of a case that you can generalize to the world's population (of this variety) of apples from these ten, anyway.

You haven't shown the marginal distribution of PL, but from the (very) limited data listing—as well as from the fitted cut points—you might consider consolidating a few of them. Or using mixed inasmuch as you have seven categories.

[Laura Andreea]

Thank you very much for your help!

[Weiwen Ng]

I don't know what Stata you are using, but I suspect that numerical overflow isn't because of it.

It's not clear from how an apple possesses an explicit attitude as a quantitative attribute, but I assume that the value for each apple's ea is identical across participants and similarly for each participant's ia across apples. It's possible that the numerical difficulty arises in teasing apart the (orthogonal) fixed and random effects, especially with apples, where there are only ten of them. (Note the magnitude of the variance estimate for apple in the model that you were able to fit.) You might want to drop the random effect for apple and just consider it fixed only. I kind of doubt that you're going to be able to build much of a case that you can generalize to the world's population (of this variety) of apples from these ten, anyway.

You haven't shown the marginal distribution of PL, but from the (very) limited data listing—as well as from the fitted cut points—you might consider consolidating a few of them. Or using mixed inasmuch as you have seven categories.

I was wondering about dropping apple as a random effect as well. Empirically, Laura could consider selecting 2 or 3 random subsets of the data, running -meologit- with apple as a crossed random effect, and look at the level 3 ICCs. If they're both as small as demonstrated above, then she'd be on reasonable empirical grounds to drop it.

I can't speak as clearly to the substantive grounds for including or dropping apple as a crossed random effect. I think Joseph's explanation is pretty reasonable.

I did a Google search for this error, and I wasn't able to find anything informative. I suspect this may come up occasionally with crossed random effects. If I were in Laura's situation and I were determined to at least attempt to fit a proper crossed effect model, I'd approach Stata's tech support and ask if they can clarify. They'll likely need a log file at minimum. Laura, don't feel obliged to do this, but if you do, I hope you can post the response here.

[Laura Andreea]

Thank you for your help .

I'd start by fitting a 2-level model, saving the parameter estimates, then trying your 3-level model and asking Stata to start from the saved parameter estimates:

I now managed to run the model with the full sample based on the code you offerend.

meologit PL c.ea##c.ia || id:
matrix b = e(b)
meologit PL c.ea##c.ia || _all: R.apple || id:, from(b)

Mixed-effects ologit regression Number of obs = 6,080

-------------------------------------------------------------
| No. of Observations per Group
Group Variable | Groups Minimum Average Maximum
----------------+--------------------------------------------
_all | 1 6,080 6,080.0 6,080
id | 608 10 10.0 10
-------------------------------------------------------------

Integration method: laplace

Wald chi2(3) = 2873.34
Log likelihood = -6669.5663 Prob > chi2 = 0.0000
------------------------------------------------------------------------------
PL | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ea | 1.317802 .0265832 49.57 0.000 1.2657 1.369904
ia | -.4672576 .1469616 -3.18 0.001 -.755297 -.1792181
|
c.ea#c.ia | .0130422 .0188672 0.69 0.489 -.0239369 .0500213
-------------+----------------------------------------------------------------
/cut1 | 3.887783 .1949488 3.505691 4.269876
/cut2 | 5.491763 .2030047 5.093881 5.889645
/cut3 | 7.022046 .2147342 6.601174 7.442917
/cut4 | 8.525579 .2273621 8.079958 8.9712
/cut5 | 10.22779 .2428352 9.751838 10.70373
/cut6 | 11.96855 .2580049 11.46287 12.47423
-------------+----------------------------------------------------------------
_all>apple |
var(_cons)| .0767617 .0408632 .0270407 .2179071
-------------+----------------------------------------------------------------
id |
var(_cons)| 1.078323 .1034247 .8935276 1.301337
------------------------------------------------------------------------------
LR test vs. ologit model: chi2(2) = 547.07 Prob > chi2 = 0.0000

If I now try to add more explanatory variables as fixed effects such as demographics, then stata is not converging anymore.

Do you know what might be the issue?




Thank you.
/Andreea

[Laura Andreea]

And is

meologit PL c.ea##c.ia || _all: R.apple || id:

same as

meologit PL c.ea##c.ia || _all: R.id || _all: R.apple

According to stata manual both are the same, only difference is that the first is more computationally efficient. Is it correct?

Thank you.