A problem in bootstrap

[Zibin Huang]

Hi, guys,
I am trying to run a code as follows using bootstrap. The program "corand" is defined by myself. Why it always shows an error information that "literated ambiguous abbreviation
an error occurred when bootstrap executed corand, posting missing values"?
The problem should be lying in the first part before "predict X_b,xb" coz when I take this part out of the program, everything would be fine. Is there any restrictions on changing dataset when we use bootstrap command in Stata? Thanks so so much!

/* For Compliers */
program corand, rclass
version 13.0

/* Get beta_hat */
foreach var of varlist mig_either fc_age preschild matime eduy f_eduy age f_age literated f_literated prefect1-prefect247 { /* Calculate E(Y|Z) and E(X|Z) */



sum `var' if fc_sex==0 & sc_sex==0 & numchi_hh==2
gen E_`var' = `r(mean)' if fc_sex==0 & sc_sex==0 & numchi_hh==2 /* For z=0,b=2,c=2 */



sum `var' if fc_sex==0 & sc_sex==1 & numchi_hh==2
replace E_`var' = `r(mean)' if fc_sex==0 & sc_sex==1 & numchi_hh==2 /* For z=0,b=1,c=2 */


sum `var' if fc_sex==1 & sc_sex==0 & numchi_hh==2
replace E_`var' = `r(mean)' if fc_sex==1 & sc_sex==0 & numchi_hh==2 /* For z=1,b=1,c=2 */


sum `var' if fc_sex==1 & sc_sex==1 & numchi_hh==2
replace E_`var' = `r(mean)' if fc_sex==1 & sc_sex==1 & numchi_hh==2 /* For z=1,b=0,c=2 */


sum `var' if fc_sex==0 & numchi_hh==1
replace E_`var' = `r(mean)' if fc_sex==0 & numchi_hh==1 /* For z=0,b=1,c=1 */


sum `var' if fc_sex==1 & numchi_hh==1
replace E_`var' = `r(mean)' if fc_sex==1 & numchi_hh==1 /* For z=1,b=0,c=1 */


}

foreach var of varlist mig_either fc_age preschild matime eduy f_eduy age f_age literated f_literated prefect1-prefect247 {

gen `var'_r=`var'-E_`var'
}


reg mig_either_r fc_age_r preschild_r matime_r eduy_r f_eduy_r age f_age_r literated_r f_literated_r prefect1_r-prefect247_r,r

drop fc_age_r preschild_r matime_r eduy_r f_eduy_r age f_age_r literated_r f_literated_r prefect1_r-prefect247_r

foreach var of varlist fc_age preschild matime eduy f_eduy age f_age literated f_literated prefect1-prefect247 {

rename `var' `var'_r

}








predict X_b,xb




gen mig_new=mig_either-X_b

count if fc_sex==0 & sc_sex==0 & numchi_hh==2
local n022 = `r(N)'
display `n022'

count if fc_sex==0 & sc_sex==1 & numchi_hh==2
local n012 = `r(N)'
display `n012'

count if fc_sex==1 & sc_sex==0 & numchi_hh==2
local n112 = `r(N)'
display `n112'

count if fc_sex==0 & numchi_hh==1
local n011 = `r(N)'
display `n011'

count if fc_sex==1 & numchi_hh==1
local n101 = `r(N)'
display `n101'

count if fc_sex==1 & sc_sex==1 & numchi_hh==2
local n102 = `r(N)'
display `n102'



sum mig_new if fc_sex==0 & sc_sex==0 & numchi_hh==2
local mig022 = `r(mean)'
display `mig022'

sum mig_new if fc_sex==0 & sc_sex==1 & numchi_hh==2
local mig012 = `r(mean)'
display `mig012'

sum mig_new if fc_sex==1 & sc_sex==0 & numchi_hh==2
local mig112 = `r(mean)'
display `mig112'

sum mig_new if fc_sex==0 & numchi_hh==1
local mig011 = `r(mean)'
display `mig011'

sum mig_new if fc_sex==1 & numchi_hh==1
local mig101 = `r(mean)'
display `mig101'

sum mig_new if fc_sex==1 & sc_sex==1 & numchi_hh==2
local mig102 = `r(mean)'
display `mig102'

sum fc_sex
local P0=1-`r(mean)'

sum sc_sex if at==1 & fc_sex==0
local Pa0=1-`r(mean)'

local nN=`n101'/(1-`P0')
local nC=(`n011'-`P0'*`nN')/`P0'
local nA=(`n022'+`n012')/`P0'



/* Bound */
local Pa1=0.871
local Pc=(`n112'-(1-`P0')*`Pa1'*`nA')/((1-`P0')*`nC')




sum X_b
local xb=`r(mean)'







local BLATEat=`mig022'-`mig012'
local BLATEco=((`mig112'+`xb')*(`Pa1'*`nA'+`Pc'*`nC')-`Pa1'*`nA'*(`mig012'+`xb'))/(`Pc'*`nC')-(((1-`Pa1')*`nA'+(1-`Pc')*`nC')*(`mig102'+`xb')-(1-`Pa1')*`nA'*(2*`mig012'-`mig022'+`xb'))/((1-`Pc')*`nC')
local BLATE=(`BLATEat'*(`nA'+`nN')+`BLATEco'*`nC')/(`nA'+`nC'+`nN')
local LATE=((`Pa1'*`nA'+`Pc'*`nC')*(`mig112'+`xb')-(`Pa1'*`nA'+`Pc'*`nN')*(`mig012'+`xb')-`Pc'*(`nN'+`nC')*(`mig011'+`xb')+`Pc'*`nN'*(`mig02 2'+`mig101'+2*`xb'))/(`Pc'*`nC')


return scalar LATE=`LATE'

return scalar BLATEco=`BLATEco'

return scalar BLATEat=`BLATEat'

return scalar BLATE=`BLATE'

return scalar Pa1=`Pa1'

return scalar Pc=`Pc'

end


bootstrap r(LATE) r(BLATEco) r(BLATEat) r(BLATE) r(Pa1) r(Pc), reps(2000) noisily nodrop: corand
estat bootstrap,all

program drop corand

[Mike Lacy]

Because of how you have posted your code, it's not possible to tell if the problems might stem from failure to use a line continuation properly. But, more importantly, your code is sufficiently complicated that it's not likely that anyone can simply inspect it and see the problem. You're going to need to try some basic diagnostic things here yourself and report back. Also, I find it hard to believe that the error message you cite "literated ambiguous abbreviation an error occurred when bootstrap executed corand, posting missing values"? is actually what Stata said. It's not a grammatical English sentence, and while Stata error messages can be cryptic, they are generally grammatical. <grin> Also, "literated" is not a word I know.

For diagnosis:
1) Check if your program "corand" works by itself, not when called by bootstrap. (You have not spoken to this issue,.)
2) Put some -display- commands inside your code and see what -corand- is doing when called by bootstrap.

[Clyde Schechter]

One more suggestion. Inside your -corand- program, put

set tracedepth 1
set trace on

right at the top and put

set trace off

at the bottom. That way if the error is occurring inside -corand-, you will be able to see which command is tripping up. And you will also be able to see how your various local macros are being expanded in real time.

[Zibin Huang]

Because of how you have posted your code, it's not possible to tell if the problems might stem from failure to use a line continuation properly. But, more importantly, your code is sufficiently complicated that it's not likely that anyone can simply inspect it and see the problem. You're going to need to try some basic diagnostic things here yourself and report back. Also, I find it hard to believe that the error message you cite "literated ambiguous abbreviation an error occurred when bootstrap executed corand, posting missing values"? is actually what Stata said. It's not a grammatical English sentence, and while Stata error messages can be cryptic, they are generally grammatical. <grin> Also, "literated" is not a word I know.

For diagnosis:
1) Check if your program "corand" works by itself, not when called by bootstrap. (You have not spoken to this issue,.)
2) Put some -display- commands inside your code and see what -corand- is doing when called by bootstrap.

Thanks Mike. Sorry for my long code. I tried to run it without bootstrap and it worked. I also tried to delete the part after “predict X_b,xb" and call bootstrap. The problem remains.
"literated ambiguous abbreviation an error occurred when bootstrap executed corand, posting missing values" is the error message and "Literated" is a variable name.

[Zibin Huang]

One more suggestion. Inside your -corand- program, put

set tracedepth 1
set trace on

right at the top and put

set trace off

at the bottom. That way if the error is occurring inside -corand-, you will be able to see which command is tripping up. And you will also be able to see how your various local macros are being expanded in real time.

Hi Clyde, I tried your suggestion and it seems that the problem lies in the first part. I changed the name of some variables at each draw and in bootstrapping, there are conflicts in the draws after the first one.

[Zibin Huang]

/* Estimation */
program corand, rclass
version 13.0

use "C:\Users\lenovo\Dropbox\My documents\Rochester\Third Year Paper\Boy Premium\2.19\Data Generated\temp.dta",clear


/* Get beta_hat */
foreach var of varlist mig_either fc_age preschild matime eduy f_eduy age f_age literated f_literated prefect1-prefect247 { /* Change migration measure this line */

/* Calculate E(Y|Z) and E(X|Z) */

sum `var' if fc_sex==0 & sc_sex==0 & numchi_hh==2
gen E_`var' = `r(mean)' if fc_sex==0 & sc_sex==0 & numchi_hh==2 /* For z=0,b=2,c=2 */



sum `var' if fc_sex==0 & sc_sex==1 & numchi_hh==2
replace E_`var' = `r(mean)' if fc_sex==0 & sc_sex==1 & numchi_hh==2 /* For z=0,b=1,c=2 */


sum `var' if fc_sex==1 & sc_sex==0 & numchi_hh==2
replace E_`var' = `r(mean)' if fc_sex==1 & sc_sex==0 & numchi_hh==2 /* For z=1,b=1,c=2 */


sum `var' if fc_sex==1 & sc_sex==1 & numchi_hh==2
replace E_`var' = `r(mean)' if fc_sex==1 & sc_sex==1 & numchi_hh==2 /* For z=1,b=0,c=2 */


sum `var' if fc_sex==0 & numchi_hh==1
replace E_`var' = `r(mean)' if fc_sex==0 & numchi_hh==1 /* For z=0,b=1,c=1 */


sum `var' if fc_sex==1 & numchi_hh==1
replace E_`var' = `r(mean)' if fc_sex==1 & numchi_hh==1 /* For z=1,b=0,c=1 */


}

foreach var of varlist mig_either fc_age preschild matime eduy f_eduy age f_age literated f_literated prefect1-prefect247 { /* Change migration measure this line */

gen `var'_r=`var'-E_`var'
}


reg mig_either_r fc_age_r preschild_r matime_r eduy_r f_eduy_r age f_age_r literated_r f_literated_r prefect1_r-prefect247_r,r /* Change migration measure this line */

drop fc_age_r preschild_r matime_r eduy_r f_eduy_r age f_age_r literated_r f_literated_r prefect1_r-prefect247_r

foreach var of varlist fc_age preschild matime eduy f_eduy age f_age literated f_literated prefect1-prefect247 {

rename `var' `var'_r

}


predict X_b,xb



gen mig_new=mig_either-X_b

end


bootstrap , reps(2000): corand
estat bootstrap,all

program drop corand

I add this line "use "C:\Users\lenovo\Dropbox\My documents\Rochester\Third Year Paper\Boy Premium\2.19\Data Generated\temp.dta",clear"
which means I reload the dataset at each draw. It seems that this works now. But is this OK for bootstrapping in Stata? According to what I believe, this process should be that each time I reload the dataset and then Stata will draw a new subsample from it and then execute the following calculation. Am I right?

[Clyde Schechter]

I tried your suggestion and it seems that the problem lies in the first part. I changed the name of some variables at each draw and in bootstrapping, there are conflicts in the draws after the first one.

This is too vague to be helpful. And you should not be reloading the data at each all of the program in -bootstrap-.

I suggest that you strip the program -corand- down to the minimum amount of code that you can run and create the problem and then try to fix that. If you need help with that, you'll need to show the actual code and output as it runs with -set trace on-. You will also need to provide example data to test the code with.

[Zibin Huang]

This is too vague to be helpful. And you should not be reloading the data at each all of the program in -bootstrap-.

I suggest that you strip the program -corand- down to the minimum amount of code that you can run and create the problem and then try to fix that. If you need help with that, you'll need to show the actual code and output as it runs with -set trace on-. You will also need to provide example data to test the code with.

/* Estimation */
program corand, rclass
version 13.0




/* Get beta_hat */
foreach var of varlist mig_either fc_age preschild matime eduy f_eduy age f_age literated f_literated prefect1-prefect247 { /* Change migration measure this line */

/* Calculate E(Y|Z) and E(X|Z) */

sum `var' if fc_sex==0 & sc_sex==0 & numchi_hh==2
gen E_`var' = `r(mean)' if fc_sex==0 & sc_sex==0 & numchi_hh==2 /* For z=0,b=2,c=2 */



sum `var' if fc_sex==0 & sc_sex==1 & numchi_hh==2
replace E_`var' = `r(mean)' if fc_sex==0 & sc_sex==1 & numchi_hh==2 /* For z=0,b=1,c=2 */


sum `var' if fc_sex==1 & sc_sex==0 & numchi_hh==2
replace E_`var' = `r(mean)' if fc_sex==1 & sc_sex==0 & numchi_hh==2 /* For z=1,b=1,c=2 */


sum `var' if fc_sex==1 & sc_sex==1 & numchi_hh==2
replace E_`var' = `r(mean)' if fc_sex==1 & sc_sex==1 & numchi_hh==2 /* For z=1,b=0,c=2 */


sum `var' if fc_sex==0 & numchi_hh==1
replace E_`var' = `r(mean)' if fc_sex==0 & numchi_hh==1 /* For z=0,b=1,c=1 */


sum `var' if fc_sex==1 & numchi_hh==1
replace E_`var' = `r(mean)' if fc_sex==1 & numchi_hh==1 /* For z=1,b=0,c=1 */


}

foreach var of varlist mig_either fc_age preschild matime eduy f_eduy age f_age literated f_literated prefect1-prefect247 { /* Change migration measure this line */

gen `var'_r=`var'-E_`var'
}


reg mig_either_r fc_age_r preschild_r matime_r eduy_r f_eduy_r age f_age_r literated_r f_literated_r prefect1_r-prefect247_r,r /* Change migration measure this line */

drop fc_age_r preschild_r matime_r eduy_r f_eduy_r age f_age_r literated_r f_literated_r prefect1_r-prefect247_r

foreach var of varlist fc_age preschild matime eduy f_eduy age f_age literated f_literated prefect1-prefect247 {

rename `var' `var'_r

}


predict X_b,xb



gen mig_new=mig_either-X_b /* Change migration measure this line */



set trace off
end


bootstrap, reps(2000) noisily: corand
estat bootstrap,all

program drop corand

This is the code I am running now. The error message is "literated ambiguous abbreviation an error occurred when bootstrap executed corand, posting missing values"
I turn on set trace, and find the error lies in this part:

foreach var of varlist mig_either fc_age preschild matime eduy f_eduy age f_age literated f_literated prefect1-prefect247

for the second draw.
It seems that I cannot upload a dta file? Is that using upload attachments? Sorry I am a new hand here.

[Clyde Schechter]

The way to give example data is not to upload a data set but to use the -dataex- command. If you are running version 15.1 or a fully updated version 14.2, it is already part of your official Stata installation. If not, run -ssc install dataex- to get it. Either way, run -help dataex- to read the simple instructions for using it. -dataex- will save you time; it is easier and quicker than typing out tables. It includes complete information about aspects of the data that are often critical to answering your question but cannot be seen from tabular displays or screenshots. It also makes it possible for those who want to help you to create a faithful representation of your example to try out their code, which in turn makes it more likely that their answer will actually work in your data.

[Zibin Huang]

The way to give example data is not to upload a data set but to use the -dataex- command. If you are running version 15.1 or a fully updated version 14.2, it is already part of your official Stata installation. If not, run -ssc install dataex- to get it. Either way, run -help dataex- to read the simple instructions for using it. -dataex- will save you time; it is easier and quicker than typing out tables. It includes complete information about aspects of the data that are often critical to answering your question but cannot be seen from tabular displays or screenshots. It also makes it possible for those who want to help you to create a faithful representation of your example to try out their code, which in turn makes it more likely that their answer will actually work in your data.

I am using "dataex". But it says "input statement exceeds linesize limit"?

[Zibin Huang]

I am using "dataex". But it says "input statement exceeds linesize limit"?

Is that because I have too many variables?

[Clyde Schechter]

Yes. So pick the minimum subset of variables that will run with the code in question and run -dataex- with that varlist. If that's still too many variables for -dataex-, post two (or, I hope not, more) -dataex- examples with separate sets of variables that can be merged together to re-create the relevant example.