save within a loop foreach

[Jala Youssef]

Hi everyone,

I have the below code that I run for each country_origin separately. I would like to replicate this code for a large group of country_origin in one step (a simplified example for the group of country_origin could be "AFG" "ALB" "AGO" "ATG").

That's an example of the code for "AFG"

import delimited "$mydir\labels_T.txt"
drop v2 v3 v5
rename v4 v2
gen `c(obs_t)' index = _n
tempfile holding
save `holding'
drop index
rename v1 country_origin
rename v2 sector_origin
merge 1:1 _n using "$mydir\data.dta", assert(match) nogenerate
keep if country_origin=="AFG"
reshape long v, i(country_origin sector_origin) j(index)
merge m:1 index using `holding', assert(match) nogenerate
drop index
rename v1 country_destination
rename v2 sector_destination
rename v value
order value, last
sort country_origin sector_origin country_destination sector_destination
generate year=2010
save "$mydir\AFG.dta"

The code as it stands ends by saving a data file for this country_origin ("AFG" in the previous example).

I thought that one way to replicate the code would be to create a local with the different country_origin and then run a loop like this for example:

countries "AFG ALB AGO ATG"
foreach x of local countries{
}

My problem is that the code would get blocked at the save step. I need to save one separate dta file for each country_origin.

I would be grateful if you could advise.

Many thanks in advance.

[Clyde Schechter]

Try this:

local countries AFG ALB AGO ATG
foreach c of local countries {}
    import delimited "$mydir\labels_T.txt", clear
    drop v2 v3 v5
    rename v4 v2
    gen `c(obs_t)' index = _n
    tempfile holding
    save `holding'
    drop index
    rename v1 country_origin
    rename v2 sector_origin
    merge 1:1 _n using "$mydir\data.dta", assert(match) nogenerate
    keep if country_origin=="`c'"
    reshape long v, i(country_origin sector_origin) j(index)
    merge m:1 index using `holding', assert(match) nogenerate
    drop index
    rename v1 country_destination
    rename v2 sector_destination
    rename v value
    order value, last
    sort country_origin sector_origin country_destination sector_destination
    generate year=2010
    save "${mydir}/`c'.dta"
}

Note, in particular, the modifications made to the -save- command. In particular, the use of backslash (\) characters as path separators (common in Windows) creates problems when the next part of the pathname is given by a local macro. In particular, the character sequence \` is not understood by Stata as a backslash followed by the start of a reference to a local macro. Rather, \` is understood as an "escape sequence" representing a literal backtick (`) character which does not begin a local macro reference. By using the forward slash (/) as the path separator the problem is overcome. Even though Windows generally defaults to \ as the path separator, it understands / in these contexts anyway, so no problem is created. (In fact, when writing programs that might ultimately be ported to a Unix or Mac system, it is best to use / exclusively as the path separator, not only to avoid confusion with local macros, but also because the / path separator is fully compatible with all of the major operating systems.)

I have also put curly braces around -mydir- to reinforce the fact that only mydir, and nothing that follows it, is intepreted as the name of a global macro to be referenced here. In this particular case, it is not really necessary: / is never interpreted as part of a global macro name. But in general when you are mixing a global macro name and other text in a sequence that is not delimited by whitespace, it is safer to explicitly tell Stata exactly what is part of the global macro name and what isn't. (Note that no such precautions are needed with local macros because references to them are always demarcated by a right quote ('). But global macro names have no final delimiter, only the initial $.)

Finally, I'll just point out that there is no reason to use a global macro to hold the name of a directory. You avoid these complications, as well as the dangers that arise from any use of global macros, by using local macros instead. I'll spare you my lengthy rant against global macros, and leave it there.

Added: I noticed Mike Lacy's #3 shortly after I posted this. He makes a number of really good comments regarding aspects of your code that I overlooked. I was just focused on making a loop and the minimum changes to your code needed for that purpose. But all of his suggestions are really excellent and following his advice will greatly improve the transparency and maintainability of your code.

[Mike Lacy]

(Substantially edited to avoid duplication with Clyde's response, which crossed with mine.)

1) I'm guess that "$mydir\labels_T.txt" is a file that contains all of the countries of origin. If I'm right, I'd think that much of the top part of the code could/should be outside the loop? Also, I wonder if the same might apply to the -reshape- and -merge- commands, but in modified form? (Perhaps I'm misunderstanding, in which case please ignore this.)

2) Isn't there a need for a -restore- at the bottom of the loop #2?

3) Some unsolicited advice: Merging by _n is usually an undesirable (dangerous?) choice because: a) Files can easily get out of physical order; b) Using _n rather than some variable with content obscures the meaning of what the merge is doing. I am also confused by creating an "index" related to _n. I would suggest rewriting the merge commands to use a meaningful key variable instead.

[Jala Youssef]

Dear Clyde Schechter and Mike Lacy

Thank you so much for your very detailed, useful and constructive guidance. This is very much appreciated at my end.

Jala