Multilevel Analysis: Crosslevel Interaction with individual level interaction

[Leonie Steckermeier]

Dear all,

I've gotten myself in terrible trouble with a very complicated model specification which includes an individual-level interaction tested for 36,000 Individuals nested in 33 countries for which I would like to know whether it varies between two country groups.

For the first step I am estimating the individual-level main effects and interaction term on the dependent variable with random slopes for main effects and the interaction:

mixed dv main1 main2 main1Xmain2 || country: main1 main2 main1Xmain2, cov(un) mle

and I am pretty sure that I need to have random slopes for both main effects and the interaction because later on I want to know whether the interaction is stronger in the two country sub-groups (e.g. richer societies vs poorer societies).

For this second step my first approach was to simply split up the sample in two groups (e.g. 17 poor countries and 16 rich countries) and rerun the same model for the two groups:

mixed dv main1 main2 main1Xmain2 if poor==0 || country: main1 main2 main1Xmain2, cov(un) mle
mixed dv main1 main2 main1Xmain2 if poor==1 || country: main1 main2 main1Xmain2, cov(un) mle

However, I have no idea how to compare whether the interaction effects differ significantly because I don't know how to calculate confidence intervals for interactions with random slopes (is that even possible?).

My second approach thus was to estimate a three way interaction - including the country level variable main3 (which I used above to split the sample in poor==0 and poor==1) - following the UCLA instruction (https://stats.idre.ucla.edu/stata/fa...tion-stata-12/) which then looked like this:

mixed dv main1 main2 main3 main1Xmain2 main1Xmain3 main2Xmain3 main1Xmain2Xmain3 || country: main1 main2 main1Xmain2, cov(un) mle

and did not converge (Hessian is not negative semidefinite).

Anyway I would not have prefered this latter model because I think three-way-interactions are terrible to interpret.

I would very much like to stick with my first approach (comparing two country sub-samples) but therefore I would need some way of comparing whether the whole interaction effect in the one group is significantly different from that in the other group.

Any ideas are highly welcome!

Thank you all very much in advance for your input!
Leonie

[Clyde Schechter]

Well, I'm not sure why you ended up with that non convergence problem. But it seems to me that your code for that is incorrect in any case. To do what you say you want I see no reason to make a main1Xmain2Xmain3 interaction. You need a poorXmain1Xmain2 interaction term. I agree that, in general, interpreting three way interactions is difficult, but in your situation where the purpose is quite specifically to see about the difference between the main1Xmain2 interaction between poor and rich countries. The coefficient of that variable will be precisely that difference.

If you also run into convergence difficulties with this approach, then I think you need to simplify the model. The first thing I would do is remove the random slope for main1Xmain2 and see if you get convergence that way.

[Leonie Steckermeier]

Dear Clyde,
thank you so much for your advice!

Using the poor-dummy (instead of the GDP of the country) for the three-way-interaction worked:

mixed dv main1 main2 poor main1Xmain2 main1Xpoor main2Xpoor main1Xmain2Xpoor || country: main1 main2 main1Xmain2, cov(un) mle

However, Stata did not produce standard errors and confidence intervals for the random parameters of the model (see below). This has never happened to me before and I don't know what it means. Is this bad? Do I need to reject the model?

satisfaction	Coef.	Std. Err.	z	P>|z|	[95% Conf.	Interval]
main1	0.5482467	0.0704094	7.79	0	0.4102468	0.6862466
main2	1.750388	0.0915953	19.11	0	1.570864	1.929911
poor	1.154255	0.4085078	2.83	0.005	0.3535939	1.954915
main1Xmain2	-0.0642548	0.0271553	-2.37	0.018	-0.1174782	-0.0110313
poorXmain1	0.1608563	0.1057663	1.52	0.128	-0.0464419	0.3681545
poorXmain2	-0.1258826	0.1346086	-0.94	0.35	-0.3897107	0.1379454
poorXmain1Xmain2	-0.0655579	0.0396629	-1.65	0.098	-0.1432956	0.0121799
_cons	0.7029876	0.2748405	2.56	0.011	0.1643101	1.241665
Random-effects Parameters	Estimate	Std. Err.	[95% Conf. Interval]
country: Unstructured
var(main1)	0.0383215	.	. .
var(main2)	0.0742891	.	. .
var(main1Xmain2)	0.0049772	.	. .
var(_cons)	0.9119286	.	. .
cov(main1, main2)	0.0311581	.	. .
cov(main1,main1Xmain2)	-0.0129428	.	. .
cov(main1,_cons)	-0.0980551	.	. .
cov(main2,main1Xmain2)	-0.0128259	.	. .
cov(main2,_cons)	-0.2257087	.	. .
cov(main1Xmain2,_cons)	0.0323958	.	. .
var(Residual)	3.178129	.	. .

Thank you so much again for your help!

[Gobinda Natak]

My impression is that you have a super-complex, probably impossible to interpret set of variance and covariance components. Why don't you start with a simpler model and check how including additional variance/covariance terms improves model fit or changes whatever you're substantively interested in? Or alternatively/additionally, think about the meaning of the variance/covariance terms you're estimating and spell out for yourself why you would be interested in them.

Good luck!
Gobinda

[Leonie Steckermeier]

Dear Gobinda,
thank you for your answer. Unfortunately this is already the easiest model for what I want to know. From what I can see, the only thing left now is either losing the main1Xmain2 random slope (which would make it impossible to test my assumption) or to estimate with a covariance type other than unstructured, for which I have no hypothesis. Thank you anyway.