LR test and ICC to decide whether to use multi-level data

[Asma Tabassum]

Hi all, I had a small inquiry and looking forward to someone helping me out!

I have a multi-level data, with individuals nested in countries, over 8 different waves.

In order to check whether to run a random-effects (multi-level model) with a random intercept or to just run a fixed effects model, I ran a random effects null model :

meprobit preference || COUNTRY:

the ICC value obtained is 0.08, so I am wondering if it indicates the need to use multi-level model?
the LR test vs probit is also significant.

However, i also ran the full model using random effects using the code:

meprobit preference age gender variablex1 variable x2 i.YEAR || COUNTRY:

and the LR test vs probit for this model shows a Prob>chibar2 = 1.000

which would mean there is no need to use a multilevel model and standard probit should work fine.


So i am confused which one to use.
In the https://www3.nd.edu/~rwilliam/Taiwan...edVsRandom.pdf file by
Richard Williams
page 31, he clearly indicates that since the LR test vs logistic model prob>chibar2 = 0, it is better to use multilevel model instead of standard logit.
However, that model was not a null model and included other explanatory variables as well.


In that case should I be following the lr test of my full model or the null model ?

If I do use a fixed effects model specification controlling for year and country effects then the following should be the -Stata- command if i am not wrong, right?

probit preference age gender variablex1 variablex2 i.YEAR i.COUNTRY, vce (robust) ?

[Asma Tabassum]

my number of countries are also relatively small - 38 countries

[Clyde Schechter]

The conclusion that I would draw from the two versions of the model you ran is that the variables age gender variablex1 variable x2 i.YEAR explain all of the variation that was accounted for by the COUNTRY level intercepts in the simpler model. This sort of thing happens now and then. So if you want to run the model with all those variables, then you can, indeeed, use -probit- for that. That says nothing about the suitability of -probit- for any model with other variables.

If I do use a fixed effects model specification controlling for year and country effects then the following should be the -Stata- command if i am not wrong, right?

probit preference age gender variablex1 variablex2 i.YEAR i.COUNTRY, vce (robust) ?

No, not right. The addition of i.COUNTRY to a linear model properly emulates a fixed effects model. But that is not true for non-linear models. Official Stata does not have a fixed-effects probit model estimator--it's complicated. If you run -search probit fixed effects- you will find links to some user-written commands that may do the trick. I have no experience with them so I cannot make any more specific recommendation about that.

[Asma Tabassum]

Thanks a lot Clyde! Always appreciate your input !