Problem using several variables in parallel loops

[Sonia Lewis]

Hi everyone,

I'm trying to create a loop that considers four different variables in parallel. I tested the code using levelsof with the sysauto dataset like this:

sysuse auto, clear
keep in 1/5
local i = 0
levelsof make, local(input)
di `input'
levelsof price, local(output)
di `output'
levelsof mpg, local(mpg)
di `mpg'
levelsof turn, local(turn)
di `turn'
di `input'

foreach v of local input{
    local i = `i' + 1
    local newoutput : word `i' of `output'
    di "`newoutput'"
    local newmpg : word `i' of `mpg'
    di "`newmpg'"
    local newturn : word `i' of `turn'
    di "`newturn'"
    }

The problem is that the lists returned in the local macros are of different lengths because only make and price are unique, so when I try to loop over make (stored in `input'), mpg and turn are empty for the last values of `input'. I tried using -valuesof- instead, like this:

sysuse auto, clear
keep in 1/10
local i = 0
valuesof(make)
di r(values)
local input r(values)
di `input'
valuesof(price)
di r(values)
local output r(values)
di `output'
valuesof(mpg)
di r(values)
local mpg r(values)
di `mpg'
valuesof(turn)
di r(values)
local turn r(values)
di `turn'
di `input'

foreach v of local input{
    local i = `i' + 1
    local newoutput : word `i' of `output'
    di "`newoutput'"
    local newmpg : word `i' of `mpg'
    di "`newmpg'"
    local newturn : word `i' of `turn'
    di "`newturn'"
    }

But now the problem is that the local macro created for each variable from r(values) gets updated each time r(values) is called, and the locals cannot then be used in the loop.

Can anyone suggest a way to solve this? Or another way of doing this?

Thanks!

Sonia

[Clyde Schechter]

It sounds like what you really want to do is loop over observations, not over values of variables. Something like this:

sysuse auto, clear

forvalues i = 1/`=_N' {
    display make[`i']
    display mpg[`i']
    display turn[`i']
}

Is this what you want?

[Nick Cox]

list make mpg turn

looks equivalent but better to me.

[Sonia Lewis]

Hi Clyde and Nick,

Thanks for your suggestions! What I actually want to do with this code is use it to create macros that I can continue to use in further processes. For example:

sysuse auto, clear
keep in 1/5
local i = 0
levelsof make, local(input)
di `input'
levelsof price, local(output)
di `output'
levelsof mpg, local(mpg)
di `mpg'
levelsof turn, local(turn)
di `turn'
di `input'

foreach v of local input{
    local i = `i' + 1
    local newoutput : word `i' of `output'
    di "`newoutput'"
    local newmpg : word `i' of `mpg'
    di "`newmpg'"
    local newturn : word `i' of `turn'
    di "`newturn'"
    
    gen turn40 = 1 if `newturn'==40
    }

Since `newturn' is empty, it generates a new variable turn40 with missing values. How can I get this to work?

I know I'm using a convoluted method to solve this example problem (which I could just solve with gen turn40=1 if turn==40), but I'm using the sysauto data to try and simplify the problem with my data. My actual dataset is a metadata file containing data for 200 datasets, including input filenames, output filenames, countries, and years of surveys. I want to use these metadata to create local macros that I can loop over all the input filenames, loading them in turn and renaming them with the appropriate output filename and generating countries and years of survey for each. See a sample of actual code for 3 datasets:

*
local i = 0
local input "dataset1.dta dataset2.dta dataset3.dta"
di "`input'"
local output "Kenya2014 Kenya2013 Kenya2012"
di "`output'"
local country "Kenya Kenya Kenya"
di "`country'"
local year "2014 2013 2012"
di "`year'"

foreach v of local input{
    local i = `i' + 1
    local newoutput : word `i' of `output'
    di "`newoutput'"
    local newcountry : word `i' of `country'
    di "`newcountry'"
    local newyear : word `i' of `year'
    di "`newyear'"
    
use "Users/Path/`v'", clear

gen country = "`newcountry'"
gen year = "`newyear'"

*polio vaccination*
 tokenize 0 4 6 8
 forvalues x=0/3{
 generate polio`x'_mth = (((vacc`1'y-1900)*12)+vacc`1'm)
 generate polio`x'_day = ((polio`x'_mth*30.4375)+vacc`1'd)
 generate polio`x'_wk = polio`x'_day/7
 replace polio`x'_wk = floor(polio`x'_wk/1)
 mac shift
 }

save "/Users/Path/`newoutput'.dta", replace

}

This works when I list all the values in the local, but it's tedious and error prone. I also tried using tokenize, but I have a tokenize within my loop so it knocks out the initial one. I thought this was a good solution, but I'd be glad to hear if you have better ideas.

Thanks,

Sonia

[William Lisowski]

I think Clyde's approach in post #2 is correct. You want to loop over the 200 observations in your metadata file, creating macros containing the values of the variables in that observation, and then use those macros when processing the input file.

Here's one approach, not the most elegant, but it gets the job done.

* Example generated by -dataex-. To install: ssc install dataex
clear
input str12 input str9 output str5 country int year
"dataset1.dta" "Kenya2014" "Kenya" 2014
"dataset2.dta" "Kenya2013" "Kenya" 2013
"dataset3.dta" "Kenya2012" "Kenya" 2012
end

local nds `=_N'

forvalues d = 1/`nds' {
    local input_`d' = input[`d']
    local output_`d' = output[`d']
    local country_`d' = country[`d']
    local year_`d' = year[`d']
    }

clear
forvalues d = 1/`nds' {
    local input `input_`d''
    local output `output_`d''
    local country `country_`d''
    local year `year_`d''
    display "processing: `input' `output' `country' `year'"
    // do work here
    }

processing: dataset1.dta Kenya2014 Kenya 2014
processing: dataset2.dta Kenya2013 Kenya 2013
processing: dataset3.dta Kenya2012 Kenya 2012

[Sonia Lewis]

Hi William and Clyde,

Thanks very much! The forvalues loops solved my problem.

Sonia