Change macros of a do file from a master do file

[Maren Borne]

Hello,

I want to write a master do file to open several do files and run them. For this I have created the following master code (I am omitting parts of the paths to not reveal private information):

global setup_path "C:\user\Project\Setup do"
foreach year of numlist 2013/2019 {
    cd "$setup_path/`year'"
    foreach file in "d" "h" "p" "r" {
        do "`year'_cross_project_`file'_ver_2023_release1.do"
    }

The above code is successful in opening the right do file. However, I am encountering the problem that in each do file that I want to run there is a "Configuration setting" written where I would have to specify several locals and globals. The configuration code in every do file is as follows:

local log_file "LOG_FILENAME"
global csv_path "CSV_PATH"
local stata_file "STATA_FILENAME"

Because of this my master do file breaks because stata is "unable to change to CSV_PATH" , since in each do file I want to run I have not specified the above macros manually.
I have tried the below code in my master do, but naturally the macros are overwritten by the generic ones that need to be configured in each do file (as the code above shows).

foreach year of numlist 2013/2019 {
    cd "$setup_path/`year'"
    foreach file in "d" "h" "p" "r" {
        local log_file "C:\user\project\Cross\Stata datasets\log_`year'_`file'"
        global csv_path "C:\user\project\Cross"
        local stata_file "C:\user\project\Cross\Stata datasets\`year'_`file'.dta"
        do "`year'_cross_project_`file'_ver_2023_release1.do"
    }
}

Is there a way I can automate changing these macros or override them from my master do? Or do I need to change them manually in each separate do file?

[DJ McArthur]

I'm not sure, but using the 'include' command instead of the 'do' command to run the individual dofiles might solve this

[Maarten Buis]

This would be a setup that might work for you. A couple of points on this example:

1. Set an absolute path only once at the top of your main.do file. After that only use relative paths. That way if you ever share your project with someone else or need to work on a different computer with a different directory structure, you can just copy your project folder, change the one cd command at the top of the main.do file, and your entire projects just runs. You do not have to hunt down every directory specified in every .do file. It is just that one command that needs to be changed and that is it. That is so wonderfully blissfully convenient, I cannot recommend it enough.
2. You can pass macros to another .do file. If you type do dofile_a.do something important, then when dofile_a.do starts running the local macros `0' `1' and `2' will exist containing "something important", "something", "important" respectively. You can use the args command to "rename" the macros `1', `2', etc.
3. If you use include instead of do the local macros will continue to exist. It acts as if the "included" do file was actually a regular part of the dofile that contained the include command.
4. Be careful with macros in file names. The \ can be part of a path, but it is also an "escape character", i.e. the \ in \`a' means "don't evaluate the macro `a'". You can see how that can mess up a path. Instead use / before a macro.

As an aside, the name master.do is now often frowned upon due to its slavery connotation. Using a name like main.do serves the same purpose without that connotation.

*---------------- main.do -------------------------
clear all
macro drop _all
frames reset

cd "c:\user\Project"

do Setup.do
foreach year of numlist 2013/2019 {
    foreach file in "d" "h" "p" "r" {
        do `year'/`year'_cross_project_`file'_ver_2023_release1.do `year' "`file'"
    }
}
*----------------- end main.do --------------------

*----------------- settings.do --------------------------
args year file

local log_file   "Cross\Stata datasets/log_`year'_`file'"
local csv_file   "Cross/`year'_cross_project_`file'_ver_2023_release1.csv"
local stata_file "Cross\Stata datasets/`year'_`file'.dta"

*----------------- end settings.do ----------------------

*-------- 2013_cross_project_d_ver_2023_release1.do ---------
args year file

capture log close
log using "`log_file'", replace

include settings.do `year' "`file'"

import delimited "`csv_file'"

// do stuff

save "`stata_file'", replace
log close
*-------- end 2013_cross_project_d_ver_2023_release1.do ---------