Need Help With 3 level model with interactions ( - mixed- using Stata 15)

[Wendy Wynne]

I am working on a research project that predicts that country-level variables will predict individual-level perceptions. I’ve never used a multi-level model and I could really use some advice to make sure I’m doing things correctly!

I have collected survey data from respondents across 18 countries, with each respondent reporting their perceptions (4 items that I average to 1 factor called “Perception”) across a range of different scenarios.
At level 1, the DV (Perception) and four predictor variables (Medium, Large, (Small is omitted), Self, and QUI are measured at the Respondent-Scenario level.
At level 2, I have a factor variable for each Respondent (Because of different response rates, each respondent provided perceptions on between 5 and 18 scenarios—I’ve dropped any observations where they reported on fewer than 5 scenarios.)
At level 3, I have 5 country-level variables (each Respondent is embedded in a single country), which are measured using archival data. These are IN, PD, UA, MA, and Tight. (There’s a sixth Level-3 variable I’d like to add, but I’m having trouble getting the model to converge so have omitted it for now.)
Ideally, I would like to test 4 of the country-level variables as an interaction with one of the level-1 variables (QUI), with the prediction that QUI predicts Perception as a main effect, and that each of the country-level variables (IN, PD, UA, MA) affect the nature of that relationship. Moreover, theory would suggest that the fifth country-level variable (Tight) might influence the magnitude of each of those four relationships (i.e., three-way interactions).
I’m willing to collect more data, but it is expensive and so I’d like to make sure I understand the empirical model before I do so—to make sure I’m collecting the right data. In that regard, I have several questions.

1. I’m new to mixed level modeling. Is the syntax of the model (copy/pasted below) correct for my objective? I don’t understand why Stata notes that there are 345 Level-3 groups (exactly the same as the Level 2 Groups). When I -tab- the variable, I get a list of 30 countries.
2. I was thinking that a different approach would be to consider only the data for which QUI==1 and forget about the interaction effects, but I wasn’t sure I could put country-level variables into Level-1 of the model without the interaction effect with a Level-1 variable. Is this possible, and what would the syntax look like? (Hypotheses could make sense either way, and this would eliminate the need for the three-way interaction.)
3. When interpreting the data for hypothesis tests, do I simply focus on the main regression table? Or do the data for the Random Effects Parameters matter in terms of assessing hypotheses (assuming standard errors are fully reported by Stata?)
4. My biggest problem right now is that often times models don’t converge. Or they converge but do not report Level-3 standard errors. In terms of additional data collection, my aim would be to collect data from respondents from more countries to come close to 50 countries represented at Level 3. Am I correct in assuming that should reduce this problem?
5. I am posting code below using the [CODE] tag… hopefully I am doing it correctly! I copied the code from the model without the country-level Tight variable or 3-way interaction(s). (When I add Tight to the model (as an additional interaction with one of the Level-1 interactions), the model converges but does not display standard errors.) Any advice would be greatly appreciated!

[code]
. mixed Perception Medium Large Self QUI##c.UA QUI##c.MAS QUI##c.PD QUI##c.IN || ParticipantIDEncoded: || EncodedCountryLiving : c.MA c.UA c.PD c.IN c.Tight

Performing EM optimization:

Performing gradient-based optimization:

Iteration 0: log likelihood = -4917.4528
Iteration 1: log likelihood = -4913.7166 (not concave)
Iteration 2: log likelihood = -4911.1487
Iteration 3: log likelihood = -4904.5822 (not concave)
Iteration 4: log likelihood = -4901.1288 (not concave)
Iteration 5: log likelihood = -4900.5816
Iteration 6: log likelihood = -4899.9071
Iteration 7: log likelihood = -4898.9582
Iteration 8: log likelihood = -4898.5918
Iteration 9: log likelihood = -4898.569
Iteration 10: log likelihood = -4898.5633
Iteration 11: log likelihood = -4898.5629
Iteration 12: log likelihood = -4898.5628

Computing standard errors:

Mixed-effects ML regression Number of obs = 4,176

-------------------------------------------------------------
| No. of Observations per Group
Group Variable | Groups Minimum Average Maximum
----------------+--------------------------------------------
Participan~d | 345 5 12.1 18
EncodedCou~g | 345 5 12.1 18
-------------------------------------------------------------

Wald chi2(12) = 65.63
Log likelihood = -4898.5628 Prob > chi2 = 0.0000

------------------------------------------------------------------------------
Perception | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
Medium | -.0386188 .030414 -1.27 0.204 -.0982292 .0209916
Large | -.0299678 .0376828 -0.80 0.426 -.1038247 .0438891
Self | -.0109311 .0300319 -0.36 0.716 -.0697925 .0479303
1.QUI | -.3943429 .3550538 -1.11 0.267 -1.090236 .3015498
UA | .0013038 .0022572 0.58 0.564 -.0031203 .0057279
|
QUI#c.UA |
1 | -.0025959 .0017444 -1.49 0.137 -.0060148 .000823
|
MAS | -.0031465 .0030996 -1.02 0.310 -.0092215 .0029286
|
QUI#c.MAS |
1 | -.00458 .0024491 -1.87 0.061 -.0093801 .0002202
|
PD | .0099201 .0045963 2.16 0.031 .0009116 .0189286
|
QUI#c.PD |
1 | .0121409 .0037061 3.28 0.001 .004877 .0194048
|
IN | .008452 .0039308 2.15 0.032 .0007478 .0161562
|
QUI#c.IN |
1 | .0015094 .003118 0.48 0.628 -.0046016 .0076205
|
_cons | -.9229564 .4384154 -2.11 0.035 -1.782235 -.063678
------------------------------------------------------------------------------

------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
Participan~d: Identity |
var(_cons) | 4.82e-08 1.19e-07 3.78e-10 6.15e-06
-----------------------------+------------------------------------------------
EncodedCou~g: Independent |
var(MAS) | 2.51e-15 2.69e-12 0 .
var(UA) | 2.40e-07 6.67e-06 4.74e-31 1.21e+17
var(PD) | .0000457 .0000124 .0000268 .0000778
var(IN) | .0000108 5.12e-06 4.28e-06 .0000273
var(Tight) | 4.51e-14 9.84e-14 6.29e-16 3.24e-12
var(_cons) | 4.81e-08 .0000239 0 .
-----------------------------+------------------------------------------------
var(Residual) | .532151 .0123901 .5084127 .5569977
------------------------------------------------------------------------------
LR test vs. linear model: chi2(7) = 773.81 Prob > chi2 = 0.0000

Note: LR test is conservative and provided only for reference. [code]

[Clyde Schechter]

Before considering your other questions, I have a concern about your syntax in the -mixed- commands. In your description you state that respondents are nested in countries, and scenarios are nested in respondents. In your code, there is no mention of a respondent level, although you do have something you call ParticipantIDEncoded as a level. So is ParticipantIDEncoded an identifier for respondents? If not, then you have introduced some new level not described and have omitted the respondent level from your model. More likely, ParticipantIDEncoded is your variable name for the respondent. In that case, you have the levels in the wrong order in your command: what you have written shows countries nested in participants. That would likely account for some of the anomalies in the results that you are questioning, including convergence issues.

Next there is the question of representing the interactions. You have stated that MA, UA, PD, IN, and Tight are country-level variables. I assume you mean that these variables take on a single value for any given country. This is not a reason to list them at the EncodedCountryLiving level in the model. By listing them there, you are asking Stata to model a random slope at the country level for each of those variables, which would mean that the effect of each of those variables on Perception is different from one country to the next: it is an interaction between Country and those variables that you are specifying. Given that the variables are defined at the country level, that doesn't make sense, and this, too, can cause the model to fail to converge.

Next there is the handling of Tight. It appears in your model as a random slope at the EncodedCountryLiving: level but is not mentioned among the fixed effects at the bottom level. That is always an error unless you specifically want to constrain the average effect of Tight to 0. Since you don't say as much, I assume you don't wish to have that constraint. You also say that you think it serves as an effect modifier of the four other country level variables and their interaction with QUI. So that would be a three way interaction indeed.

Since you are using factor-variable notation anyway, I would recommend that you apply that to the small/medium/large category variable as well. Just make a single three level variable, let's call it size for the sake of discussion, and include i.size in the model rather than Medium and Large. Also, you may as well group UA MAS PD and IN together--this will make the code more compact and readable. These final change are just notational and of a cosmetic nature, and they certainly won't change the convergence of the models. So, my suggestion is:

mixed Perception i.size i.QUI##c.(UA MAS PD IN)##c.Tight || EncodedCountryLiving: || ParticipantIDEncoded:

[Wendy Wynne]

Trying again to post the formatted code!

. mixed Perception Medium Large Self QUI##c.UA QUI##c.MAS  QUI##c.PD QUI##c.IN    || ParticipantIDEncoded: || EncodedCountryLiving :  c.MA c.UA  c.PD c.IN c.Tight

Performing EM optimization: 

Performing gradient-based optimization: 

Iteration 0:   log likelihood = -4917.4528  
Iteration 1:   log likelihood = -4913.7166  (not concave)
Iteration 2:   log likelihood = -4911.1487  
Iteration 3:   log likelihood = -4904.5822  (not concave)
Iteration 4:   log likelihood = -4901.1288  (not concave)
Iteration 5:   log likelihood = -4900.5816  
Iteration 6:   log likelihood = -4899.9071  
Iteration 7:   log likelihood = -4898.9582  
Iteration 8:   log likelihood = -4898.5918  
Iteration 9:   log likelihood =  -4898.569  
Iteration 10:  log likelihood = -4898.5633  
Iteration 11:  log likelihood = -4898.5629  
Iteration 12:  log likelihood = -4898.5628  

Computing standard errors:

Mixed-effects ML regression                     Number of obs     =      4,176

-------------------------------------------------------------
                |     No. of       Observations per Group
 Group Variable |     Groups    Minimum    Average    Maximum
----------------+--------------------------------------------
   Participan~d |        345          5       12.1         18
   EncodedCou~g |        345          5       12.1         18
-------------------------------------------------------------

                                                Wald chi2(12)     =      65.63
Log likelihood = -4898.5628                     Prob > chi2       =     0.0000

------------------------------------------------------------------------------
  Perception |      Coef.   Std. Err.      z         P>|z|                   [95% Conf. Interval]
-------------+----------------------------------------------------------------
      Medium |  -.0386188    .030414    -1.27   0.204    -.0982292    .0209916
       Large   |  -.0299678   .0376828    -0.80   0.426    -.1038247    .0438891
        Self     |  -.0109311   .0300319    -0.36   0.716    -.0697925    .0479303
       1.QUI   |  -.3943429   .3550538    -1.11   0.267    -1.090236    .3015498
          UA    |   .0013038   .0022572     0.58   0.564    -.0031203    .0057279
             |
    QUI#c.UA |
          1        |  -.0025959   .0017444    -1.49   0.137    -.0060148     .000823
             |
         MAS    |  -.0031465   .0030996    -1.02   0.310    -.0092215    .0029286
             |
   QUI#c.MAS |
          1        |    -.00458   .0024491    -1.87   0.061    -.0093801    .0002202
             |
          PD     |   .0099201   .0045963     2.16   0.031     .0009116    .0189286
             |
    QUI#c.PD |
          1        |   .0121409   .0037061     3.28   0.001      .004877    .0194048
             |
          IN      |    .008452   .0039308     2.15   0.032     .0007478    .0161562
             |
    QUI#c.IN |
          1        |   .0015094    .003118     0.48   0.628    -.0046016    .0076205
             |
       _cons  |  -.9229564   .4384154    -2.11   0.035    -1.782235    -.063678
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
Participan~d: Identity       |
                  var(_cons) |   4.82e-08   1.19e-07      3.78e-10    6.15e-06
-----------------------------+------------------------------------------------
EncodedCou~g: Independent    |
                    var(MAS) |   2.51e-15   2.69e-12             0           .
                     var(UA) |   2.40e-07     6.67e-06      4.74e-31        1.21e+17
                     var(PD) |   .0000457     .0000124      .0000268    .0000778
                     var(IN)  |   .0000108     5.12e-06      4.28e-06       .0000273
                  var(Tight) |   4.51e-14     9.84e-14      6.29e-16        3.24e-12
                  var(_cons) |   4.81e-08   .0000239             0           .
-----------------------------+------------------------------------------------
               var(Residual) |    .532151   .0123901      .5084127    .5569977
------------------------------------------------------------------------------
LR test vs. linear model: chi2(7) = 773.81                Prob > chi2 = 0.0000

Note: LR test is conservative and provided only for reference.

[Clyde Schechter]

I'm guessing that O.P. did not see #2 before posting #3. The model shown in #3 is ill-formed and the results are not interpretable, as explained in #2.

[Wendy Wynne]

Thank you so much for your lightning fast response! I clearly have a lot to learn (and yes, I didn't realize you'd answered when I reposted results!) I will make these adjustments and circle back when I have some more valid results. Thank you!