mi passive and generating a var

[Hiro farabi]

Hi all,

I have imputed my database and now i have a var which included 4 type of z and 4 type of c vars (from reshape before imputation) . I am trying to generate a var by sum up the value of some rows. as below

renam eq5dutility_b (x1)
rename eq5dutility_6m (x2)
rename eq5dutility_12m (x3)
rename eq5dutility_18m (x4)
rename tota_C_without_b (x5)
rename tota_C_without_b_6 (x6)
rename tota_C_without_6_12 (x7)
rename tota_C_without_12_18adj (x8)

reshape long x, i(pid) j(type0)

mi set mlong
mi register imputed x
mi register regular age ethnic recruited geoarea marital employment severity site cid

mi impute chained (regress) x = age ethnic recruited geoarea marital employment severity site cid , add(2) rseed(12345) by(group)

*mi passive: gen total_qaly = QALYb_6m+ QALY6_12m +QALY12_18m_disc

mi register passive qaly_at18
mi passive: by pid: gen qaly_at18 = ((x+x[_n-1])/2*(6/12)) if type0== 2 + ((x+x[_n-1])/2*(6/12)) if type0== 4 + ((x+x[_n_1])/2* (6/12)/ (1+0.035)) if type0 ==6

But the last line doesn't work. could you help me to refine it?

thank you for your suggestions.

Best wishes,
Hiro

[Clyde Schechter]

mi passive: by pid: gen qaly_at18 = ((x+x[_n-1])/2*(6/12)) if type0== 2 + ((x+x[_n-1])/2*(6/12)) if type0== 4 + ((x+x[_n_1])/2* (6/12)/ (1+0.035)) if type0 ==6

Your problem has nothing to do with -mi passive-. This command will not work even in the context of a non-mutiply-imputed data set. Stata syntax does not allow:

gen variable = this if some_condition + that if another_condition + another_thing if still_another_condition

Stata syntax is far more restrictive than English.

To do this you must* break the command up:

gen variable = this if some_condition
replace variable = that if another_condition
replace variable = another_thing if still_another_condition

To do this for a passive variable in a multiply imputed data set, each of these commands can be prefixed with -mi passive:-. And of course, the -by- prefix can be used as well.

That said, passively generated variables should be used with caution in multiple imputation. It is usually better to create the variable in the unimputed data first, allowing it to have missing values when the computation is not possible, and then multiply impute to fill in those missing values.

*I lied. There is a way to do this in a single command. But it cannot be done by using plus signs to string together separate clauses with -if- conditions. It can be done using nested calls to the -cond()- function. This approach is more compact when there are only two conditions involved. But with three or more conditions, the resulting code becomes confusing to read unless you take pains to clarify it with unconventional formatting choices, so I instead recommend a separate command for each condition as shown.

[Hiro farabi]

Your problem has nothing to do with -mi passive-. This command will not work even in the context of a non-mutiply-imputed data set. Stata syntax does not allow:

gen variable = this if some_condition + that if another_condition + another_thing if still_another_condition

Stata syntax is far more restrictive than English.

To do this you must* break the command up:

gen variable = this if some_condition
replace variable = that if another_condition
replace variable = another_thing if still_another_condition

To do this for a passive variable in a multiply imputed data set, each of these commands can be prefixed with -mi passive:-. And of course, the -by- prefix can be used as well.

That said, passively generated variables should be used with caution in multiple imputation. It is usually better to create the variable in the unimputed data first, allowing it to have missing values when the computation is not possible, and then multiply impute to fill in those missing values.

*I lied. There is a way to do this in a single command. But it cannot be done by using plus signs to string together separate clauses with -if- conditions. It can be done using nested calls to the -cond()- function. This approach is more compact when there are only two conditions involved. But with three or more conditions, the resulting code becomes confusing to read unless you take pains to clarify it with unconventional formatting choices, so I instead recommend a separate command for each condition as shown.

thank you so much for your response. i am really confused. it was suppose to do


mi set mlong
mi register imputed eq5dutility_b eq5dutility_6m eq5dutility_12m eq5dutility_18m
mi register imputed tota_C_without_b tota_C_without_b_6 tota_C_without_6_12 tota_C_without_12_18adj
mi register regular age ethnic recruited geoarea marital employment severity site cid

mi impute chained (regress) eq5dutility_b eq5dutility_6m eq5dutility_12m eq5dutility_18m tota_C_without_b tota_C_without_b_6 tota_C_without_6_12 tota_C_without_12_18adj = age ethnic recruited geoarea marital employment severity site cid , add(2) rseed(12345) by(group)

mi passive: gen qaly_at18month = ((eq5dutility_6m+eq5dutility_b)/2 *(6/12)) + ((eq5dutility_12m+eq5dutility_6m)/2 *(6/12)) + ((eq5dutility_18m+eq5dutility_12m)/2 *(6/12)/(1+0.035))

mi passive: gen total_cost =tota_C_without_b_6+ tota_C_without_6_12+ tota_C_without_12_18adj

rename total_cost y1
rename qaly_at18month y2
mi reshape long y , i(pid) j(type)
label values type
gen cost=type==1
egen typegroup = group(type group )
mi estimate :mixed y i.cost i.cost#i.group i.cost#c.tota_C_without_b i.cost#c.eq5dutility_b || site: || cid:group , reml residuals(independent, by(typegroup))

but because of reshaping befor estimation the estimation dont convergnce. i am going to reshape befor doing imputation and then do imputation. but I have strugle with mi passive: gen qaly_at18month = ((eq5dutility_6m+eq5dutility_b)/2 *(6/12)) + ((eq5dutility_12m+eq5dutility_6m)/2 *(6/12)) + ((eq5dutility_18m+eq5dutility_12m)/2 *(6/12)/(1+0.035))
in new way to do.

[Clyde Schechter]

What does "but I have strugle with" mean? What is going wrong? Is Stata giving you an error message with this command? Or is it producing results you are not expecting?

Please post back showing not just this command, but any output that Stata gives you in the Results window, and, if it produces unexpected results, show the results it produces and also what you expected to get.

[Hiro farabi]

What does "but I have strugle with" mean? What is going wrong? Is Stata giving you an error message with this command? Or is it producing results you are not expecting?

Please post back showing not just this command, but any output that Stata gives you in the Results window, and, if it produces unexpected results, show the results it produces and also what you expected to get.

I have got this error :


. mi estimate :mixed y i.cost i.cost#i.group i.cost#c.tota_C_without_b i.cost#c.eq5dutility_b || site: || cid:group , reml
> residuals(independent, by(typegroup))
convergence not achieved
model did not converge on m=1
r(498);

end of do-file

r(498);

[Clyde Schechter]

So I see several potential problems with this command that predispose it to not converge. What is the variable group for which you have specified a random slope at the cid: level. The name suggests that it is probably discrete, as does its inclusion with i. prefixed in the first interaction term. But you have treated it as a continuous variable and you have also, tacitly, constrained the mean slope of group to be 0, which is not typically what is wanted. The way you have specified the interactions is legal, but not conventional and I don't recommend it here.

The variables site and cid are used as levels in the model. The way you have specified them, you have cid's nested in sites. Is that correct, or backwards? If you have it backwards, the model will definitely not converge. How many cid's are there? How many sites? If the numbers are small, it may be best to just include them as fixed effects rather than random effects in the model. And the use of -residuals(independent, by(typegroup))- may be causing Stata to vainly try to estimate a large number of residual variances, one for each possible value of typegroup, with little or no data on each to go on--this can cause non-convergence. So how many values of typegroup are there? And how many observations are there with each value of typegroup? Unless you have a small number of values of typegroup, and a moderately large number of observations for each such value, you are better off not using this -residuals- option.

As a start, I suggest you do the following. Try the simpler code:

mixed y i.cost##(i.group c.tota_C_without_b c.eq5dutility_b) || site: || cid:i.group, reml

and just run it on the original, unimputed data set. (If it's really site nested in cid, then, of course, change the order of the random levels in the code, too.) Pay attention to the details shown in bold face. Let's see if it converges that way. Then we can go step-by-step from there. Convergence problems are difficult, and more so when the complications of multiple imputation are added into the mix. So fixing this up may turn out to be a lengthy, slow process.

[Hiro farabi]

So I see several potential problems with this command that predispose it to not converge. What is the variable group for which you have specified a random slope at the cid: level. The name suggests that it is probably discrete, as does its inclusion with i. prefixed in the first interaction term. But you have treated it as a continuous variable and you have also, tacitly, constrained the mean slope of group to be 0, which is not typically what is wanted. The way you have specified the interactions is legal, but not conventional and I don't recommend it here.

The variables site and cid are used as levels in the model. The way you have specified them, you have cid's nested in sites. Is that correct, or backwards? If you have it backwards, the model will definitely not converge. How many cid's are there? How many sites? If the numbers are small, it may be best to just include them as fixed effects rather than random effects in the model. And the use of -residuals(independent, by(typegroup))- may be causing Stata to vainly try to estimate a large number of residual variances, one for each possible value of typegroup, with little or no data on each to go on--this can cause non-convergence. So how many values of typegroup are there? And how many observations are there with each value of typegroup? Unless you have a small number of values of typegroup, and a moderately large number of observations for each such value, you are better off not using this -residuals- option.

As a start, I suggest you do the following. Try the simpler code:

mixed y i.cost##(i.group c.tota_C_without_b c.eq5dutility_b) || site: || cid:i.group, reml

and just run it on the original, unimputed data set. (If it's really site nested in cid, then, of course, change the order of the random levels in the code, too.) Pay attention to the details shown in bold face. Let's see if it converges that way. Then we can go step-by-step from there. Convergence problems are difficult, and more so when the complications of multiple imputation are added into the mix. So fixing this up may turn out to be a lengthy, slow process.

thank you for your suggestion. i dont have the same problem with unimputed dataset. i have got this error just when i have run it after imputation.