Using macros in file names (in and out of loops)

[Elizabeth Pisani]

I have two questions related to the use of macros in file names.

1) I would like to use the value of a variable in a file name. This has come up before in the forum (for example here: https://www.statalist.org/forums/for...p-in-file-name) in use cases more complex than mine, but I've been unable to make the solutions proposed work in my simple case.
I'm reading in one txt file at a time in a loop, and reducing each to a single observation.

* Example generated by -dataex-. For more info, type help dataex
clear
input str10 filename byte run str8 barcode str6 mass
"allo_059_2" 2 "allo_059" "0.1948"
end

I would like to name each of these files with the value of the variable filename (in the example given, this is "allo_059_2"), assigning it to a directory already set in my globals (in this case $tempfiles)
I tried the obvious (also with endless permutations of double quotations)

local id filename
tab `id'
save "$tempfiles/`id'_text.dta"

This returns the correct value in the tabulation, but does not read the local into the filename, giving me only the text of the varname, as below.

file /Users/elizabethpisani/stata_tempfiles/barcode_text.dta saved

I have successfully but clumsily worked around this using levelsof

levelsof filename, local(name)
foreach f of local name {
save "$tempfiles/`f'_text.dta", replace
}

but this gets messy in later, more complex nested loop transformations.

Question 2:
I need to append the data in the files I'm generating in the process above with their twin among other files which I am reading in in csv format. For example, I need to append the allo_059_2_text.dta, created in the example above, to a csv twin which is named allo_059_2.csv. I read the csv files in a loop but cannot use the same local to define the matching file to be appended because of the csv extension.

local freqfiles : dir "$freq" files "*.csv"
clear
foreach file of local freqfiles {
    clear
import delimited using `file', varnames(1)

append using "$tempfiles/`file'_text.dta"
}

predictably gives me this error: file /Users/elizabethpisani/stata_tempfiles/allo_059_2.csv_text.dta not found

I tried to kludge it by creating a local and using that in the filename

local freqfiles : dir "$freq" files "*.csv"
clear
foreach file of local freqfiles {
    clear
import delimited using `file', varnames(1)
gen fileshort = subinstr("`file'",".csv","",.)
local fs fileshort
tab `fs'
append using "$tempfiles/`fs'_text.dta"
}

again this gives me the correct readout in the tabulation of the local, but does not read it in to the file name, putting me back at the problem addressed in my first question.
For avoidance of doubt, all of the filepaths expressed through a $global include the full path to the root directory.

All help gratefully received.

[Mike Lacy]

Question 1:

local id filename

assigns the string "filename" to the local named id.

If you include the "=" sign:

local id = filename

Stata notes that "filename" is the name of a variable, and in the absence of any reference to a particular observation, assigns to that local the value of the filename variable that it finds in the *first* observation, this last feature being a default and somewhat obscure behavior of Stata. I'd rather reference the first observation explicitly:

local id = filename[1]

Question 2:
Your approach here is reasonable, but there's no need to use -fs-:

local freqfiles : dir "$freq" files "*.csv"
clear
foreach file of local freqfiles {
   clear
   import delimited using "`file'", varnames(1)
   local fileshort = subinstr("`file'",".csv","",.)   // must be a local, not a variable
   append using "$tempfiles/`fileshort'_text.dta"
}

This assumes, as I guess you know, that the global -freq- is an appropriate directory name and that the filenames you're working with dependably have the same parallel form.

[Nick Cox]

Question 1 is subtle and has bitten many users. For the story see e.g. https://journals.sagepub.com/doi/10....6867X231175349 or

Wernow J. 2005. FAQ: I have an if or while command in my program that only seems to evaluate the first observation. What’s going on? http://www.stata.com/support/faqs/pr...-if-qualifier/.

The FAQ title gives the answer rather than the usual question, which is more usually just puzzlement about what a local macro is doing, as here.

A generic debugging trick is to ask to see what is in the local macro concerned.

di "`id'" 

macro list

[Elizabeth Pisani]

Thank you, gentlemen.