Melogit and predicted probabilities for country group differences

[Tarjei W. Havneraas]

Hi

I am currently working on a multilevel logit model using melogit. The focus of the study is to compare predicted probabilities between different groups of countries. Countries are divided into five groups that corresponds to five different "welfare regimes". The analysis is based on a pooled sample of all seven waves of European Social Survey (ESS).

The question is whether my approach of obtaining predicted probabilities is correct. As I have done a lot of research to find answers on the web, I hope someone on this forum can help out.

Here is a description of the approach:

ESS data documentation states that comparisons of two or more countries or combining different countries to describe regions requires design weights in combination with population size weights (dweight*pweight=nweight). Thus, svyset for data:

svyset country [pweight=nweight], vce(linearized) singleunit(missing) || idno

I then run three different models (1) intercept-only, (2) individual-level variables, and (3) in the third and final model I add the country group (welfare regime) variable. The third model has the lowest AIC and BIC, and is therefore used in further analysis. Here is model 3:

melogit y i.x i.essround i.x i.x x x i.welfareregime || country:, or

To obtain predicted probabilities for all countries, controlling for other variables, I use the following code:

margins, at(x=(1 2 3)) post vsquish

And for country group differences:

margins, at(x=(1 2 3) welfareregime=x) post vsquish

The two main questions are (i) whether the model has been weighted accurately, and (ii) whether I should use the “svy:” prefix. When I run the models with “svy:” prefix I cannot obtain AIC or BIC scores, and I also experience problems with calculating predicted probabilities.

Best
Tarjei W. Havneraas

[Clyde Schechter]

(i) whether the model has been weighted accurately

You should consult the documentation for the ESS to answer this question. Well-curated surveys come with explanatory documents that tell you how to deal with weighting, stratification, and sampling units. Note that if the design of the ESS includes stratification and sampling units, then your -svyset- command must account for those as well. Just using the weighting will not suffice.

Your posted commands make no sense. The variable x appears 5 times in your -melogit- command, three times as a categorical variable and twice as continuous. What is that about?

Once you get a sensible -melogit- command going, if you want to look at the five welfare groups and compare them, the command would be:

margins welfareregime, pwcompare

whether I should use the “svy:” prefix.

Absolutely. If you have data from a complex survey design, analyzing it without using the -svy:- prefix just produces garbage. The reason you don't get AIC and BIC after that is because those are based on the likelihood-ratio statistic, which isn't calculable with a -svy:- based analysis. You'll simply have to live without them.

[Tarjei W. Havneraas]

Thanks for your helpful and informative reply. I will consult the data documentation again to look for descriptions regarding multilevel weighting. I apologize for the misspecification.The commands are of course meant to be stated as x1, x2, etc. Thanks for pointing this out, and for the code which should be utilized for group comparisons.