Mixed Level Models with SVY set no longer working after Stata 14 udpate!

[Elizabeth Buckner]

Hello - we are running data on multilevel survey data that is no longer working after Stata14 update.

Previous code (worked fine!) did a survey set for a PSU1 of school, with final teacher weights and BRR weights:

svyset idschool [pw = tchwgt], brr(trwgt1-trwgt100) fay(0.5)

This worked to run the following multilevel regression commands:

svy: meglm DV IV $tcontrols $scontrols || uniqueid:

Now, the most recent version of Stata (December 01) is giving a number of errors. First, it states:

survey final weights not allowed with multilevel models;
a final weight variable was svyset using the [pw=exp] syntax, but multilevel models require that each stage-level weight variable is
svyset using the stage's corresponding weight() option
an error occurred when svy executed meglm
r(459);

If I change the survey weights to align to the model in the online manual, so that we can have very basic teacher weights:

svyset idschool || _n, weight(tchwgt)

I then get the following errors:
initial values not feasible
an error occurred when svy executed meglm
r(1400);

I understand that this is a hard error to diagnose/treat. We have about 117,819 observations, with roughly 10 control variables, and a standardized DV - but dropping some does not seem to solve the issue.

It is very frustrating because we need to run multilevel models with survey weights, and this was working fine until very recently! Can anyone help or advise how to overcome this error with the current version of Stata14?

Thanks!

[Stephen Jenkins]

Do you more information about the version (update) you were using before you updated to 01 December update? If I do help whatsnew, I find that the most recent explicit reference to meglm is in the update for 05may2015. Number 11 from 22april2015 has

11. svy with multilevel models and without stage-level weights specified in svyset
failed to exit with an error when multilevel groups were not nested within the
sampling stages. The affected commands are meglm, melogit, meprobit, mecloglog,
meologit, meoprobit, mepoisson, menbreg, mestreg, and gsem. This has been fixed.

This suggests that if your previous estimates were derived with a pre-22april2015 update, you were not getting an error when you should have been -- so your previous results should be discarded, and the current error message is correct ... unless you modify the declaration of weights in your svyset statement somehow. (I don't know how to do this.)

[Elizabeth Buckner]

Thank you - this is very helpful. If I understand, then, we must have weights for each stage if we want to do multi-level. However, our dataset (for anonymity reasons) has chosen to provide only teacher level weights, combined into one, and use BRR replication weights for standard errors. We do not really have a school-level /teacher weight separate from the final teacher weight.

It does seem possible to modify the svyset command, to the following: svyset idschool || _n, weight(tchwgt)

This works well and calculates the exact same survey-weighted mean as the svyset command: svyset idschool [pw = tchwgt], brr(trwgt1-trwgt100) fay(0.5), which I take to be a good sign.

However, now I am consistently getting the following error when trying to do a random-effects model with the command:

svy: meglm DV IV $tcontrols $scontrols || uniqueid:

initial values not feasible
an error occurred when svy executed meglm
r(1400);

Does anyone have advice on how to deal with this? We believe that a two-level model is necessary.

Thanks!

[Steve Samuels]

I've not used meglm, so have little to say about the "initial values" problem. Perhaps you'll get better results with mixed, which fits the same linear mean model.

You need a brr() option to identify the replicate weights. I would guess-but you have to check- that those weights are the overall weights for sampling teachers (and students, if all student of a teacher were studied), called \(w_{ij}\) on pp 80-86 of the Survey manual. I would speculate that the teacher weight is the weight for selecting the teacher within a school (\(w_{j | i}\)= 1/Pr{selecting the teacher, given the school was selected} . If so, would be able to recover the school replicate weight as \schoolwt = replicate weight/ tw. (You need to do this for all the replicate weights) . You will know you are correct if the division yields the same value for all observations in the school.Then, you should create the new constant replicate weights (schoolwt*) Your svyset statement would be:

svyset idschool , brr(schoolwt*) || _n, weight(tchwgt)

and the meglm statement would contain no weights.

[Guillermo Montes]

I am encountering a similar error on Stata 14 (latest update). We run a multilevel model as follows:

svyset idnumr [pweigh=weight], strata(state)
svy: gsem (binaryoutcome <- $covars M(state)), logit  latent(M)

This produces an error: " survey final weights not allowed with multilevel models; a final weight variable was svyset using the [pw=exp] syntax, but multilevel models require that each stage-level weight variable is svyset using the stage's corresponding weight() option"

Since I did not use brr, what do you advice to fix this error?

Thank you,
Guillermo Montes

[Steve Samuels]

Guillermo,. You are not including a lower-level weight in your svyset statement. Elizabeth did not include an observation level weight in her svyset statement, in addition to the lack of replicate weight information.

What was your actual design? What, for example, were the primary sampling units? secondary units? What units does idnumr refer to? I don't know enough about random coefficient models in gsem to comment knowledgeably, so please explain what your model means.


Elizabeth:



Sorry, meglm will work only with survey linearized standard errors (p. 86, ME manual). However gsem, which can also fit multilevel models, will work with replicate weights (demonstrated below) and will accept stage-level weights too (p. 104, Intro 10, ME manual.)

I don't know the multilevel syntax, but Example 39g in the ME manual has an example of a three-level model for generalized responses.

I ran the following simple linear regression with gsem after svyset to check that BRR weights are okay. (The actual weights were jackknife weights.)

use http://www.stata-press.com/data/r14/stage5a_jkw, clear
svyset [pweight=pw], brrweight(jkw_*) vce(brr)
svy: gsem (yreg <-x1)

So recalculate the weights and if they look okay (school replicate weights are constant within school), try gsem with the svyset statement in my previous post .

A request: In future posts, put all commands and results between CODE delimiters, described in FAQ 12.

[Steve Samuels]

Correction: The reference to intro(10) in my last post is on page 104 of the SEM manual, not the ME manual.