Stata .do file to update other Stata .do files

[Tahlor Bolding]

I'm trying to write a script that updates globals in other scripts. I've tried using "filefilter", but I need it to find the string, and then replace that whole line with the new string, not just the matched string. I've also tried using file read/write. The only way I could figure out how to write to a specific line number was, for every line, read a line of the original script, write it to the new script if it weren't a match, otherwise write the new line. However, in doing this Stata evaluated all of my macros (i.e. blanked them out in the new file).

My next approach would be to combine the two approaches where the "file read" first looks through the script where it matches a partial line, and then passes the whole line as the string to replace argument in the "filefilter". However, I think I'll still be limited by macro substitution (e.g. I'm fine if the line I'm searching for and the replacement line have no macros, but otherwise not necessarily). Is there 1) an easier, more efficient way of doing what I'm trying to do and 2) is there a way to turn off macro substitution for an entire line?

[Clyde Schechter]

While I get the gist of what you are trying to do, I think in order to advise you, it would help to see an example of what the scripts you are trying to modify look like in the before and desired after state. And showing us the code you've tried would be helpful, too.

Please post that information in code blocks so that it is easily readable.

[Tahlor Bolding]

Example of using file read/write.

tempvar script1
tempvar script2

file open `script1' using "$directory/`file'", read
local out_file = subinstr("`file'", ".do", "_r.do", .)
capture erase "$directory/`out_file'"
file open `script2' using "$directory/`out_file'", write append

while 1 {

file read `script1' line
if r(eof)==1 & `"`line'"' == "" continue, break
if strpos(`"`line'"', "gl script_path") file write `script2' `"gl script_path $directory/`file'"' _n
else file write `script2' `"`line'"' _n

}

file close `script1'
file close `script2'

Using the file read with filefilter (where I'm getting a "Unresolved backslash escape sequence" error, which I'm guessing means I can't have any backslashes in the original script?)

tempvar script1
file open `script1' using "$directory/`file'", read
local out_file = subinstr("`file'", ".do", "_r.do", .)
capture erase "$directory/`out_file'"
disp `"gl script_path "$directory/`file'""'
while 1 {

file read `script1' line
if r(eof)==1 & `"`line'"' == "" continue, break
if strpos(`"`line'"', "gl script_path") | strpos(`"`line'"', "global script_path") filefilter "$directory/`file'" "$directory/`out_file'", from(`"`line'"') to(`"gl script_path "$directory/`file'""')

}

file close `script1'

[Clyde Schechter]

OK, but you don't show a sample of what the input file looks like and what you want the ouput file to look like. In particular, the global macro $directory is, I presume, defined somewhere before the code you show above is run since you use it to specify a path in your -file open- commands. What I can't discern is your intended action in the line

if strpos(`"`line'"', "gl script_path") file write `script2' `"gl script_path $directory/`file'"' _n

Do you want to write out the contents of $directory and the contents of local macro file into out_file? Or do you want to write the literal strings "$directory" and "`file'"? Or some combination thereof?

[Sergio Correia]

Tahlor,

What exactly are you trying to accomplish? Most scenarios I can think of would be better solved by other alternatives. For instance, doing an *include* command at the beginning of each file and defining the globals only once in the included file.

Best,
S

[Tahlor Bolding]

@Clyde Schechter: Good point. To clarify, the only feature I would want that can't be easily accomplished using backslashes would be to prevent macros from being evaluated in the line being read in. For instance, if a line reads "gl temp $temp/file", then when I search for the line in filefilter it will substitute out $temp. Then if the value of $temp is anything other than "\$temp" (which I don't think I can do), the line won't be found.

My main question is just that my solution is more of an inelegant hack, and it would surprise me if this is no way to prevent Stata from macro-evaluating lines it reads in from text files, or if there's not a more straightforward way of modifying a particular line of an ASCII file in the way I described in preceding posts.

@SergioCorreia: This is a good tip. In this instance, I want to be able to access the script name, because in the script I save out a log file as a variant of the script name (which may change). Another instance of where this could be useful: Suppose there is a bit of code that takes 1 hour to run and you need to loop through it 50 times. Empirically, I've determined that there are some efficiencies gained by running multiple sessions of Stata in parallel. Then you can imagine having some automated way of having Stata divvy up the workload into several sessions, and being able to create or edit "subscripts" would be useful.

[Clyde Schechter]

Oh, I get it. Yes, I see that the -file read- command itself evaluates the string $temp when it creates the local macro line, and that, causes $temp to be lost.

I'm not aware of any options that -file read- has that will prevent this from happening. I think you may need to do this using a general-purpose scripting language rather than in Stata. Or, maybe somebody else knows a workaround.

[Clyde Schechter]

One last thought. There may be a way to do what you want to do by reading each file into a strL variable using the fileread() function and then doing something with regular expressions to replace matches to `"gl script_path*`=char(10)'`=char(`13')"' [or whatever your OS's newline representation is] with `"gl script_path \$directory/`file'"'. Then output the result using the filewrite() function. I'd try to hack out some code, but I'm kind of a dunce when it comes to regular expression functions, and there is the complication that regexr() only replaces the first match. Since what you want to swap in will also be a match, you need to swap in some kludged-version of it that won't match and then use subinstr() to de-kludge it. But maybe this approach will work if you try it.

[Sarah Edgington]

Oh, I get it. Yes, I see that the -file read- command itself evaluates the string $temp when it creates the local macro line, and that, causes $temp to be lost.

I'm not aware of any options that -file read- has that will prevent this from happening. I think you may need to do this using a general-purpose scripting language rather than in Stata. Or, maybe somebody else knows a workaround.

I think the problem you're having is not actually that -file read- evaluates the macro. I think -file write- is where the evaluation happens. I've gotten around a similar problem in the past using macval.

I haven't tested this but I think this simple version should read a line and write it back out even if it has macro references

file read `script1' line
while r(eof)==0 {
   file write `script2'  macval(`"`line'"') _n
}

Note that if any of your dofiles contain compound quotes anywhere you'll likely run into problems. Clyde's idea of pulling the whole file into a strL may be a better idea depending on the nature of your dofiles.

[Tahlor Bolding]

@Clyde That's a good thought, more than I want to tackle right now, but probably the most robust solution.

@Sarah I think macval is exactly what I wanted. The line that seems to work is:

file write `script2'  `"`macval(line)'"' _n

You're right the double quotes are a problem however. So, to summarize my options:

1. Use file read/write method, handle macro substitution using macval, lines with double quotes are unhanded
2. Use file read/write in conjuction with filefilter, macro substitution unhanded (unless macval can be used), but lines with double quotes can be ignored (e.g. copied over unaffected).
3. Use Clyde's solution
4. Use Python

This is helpful, thanks everyone.