Calling do files, and using tempfile

[Gregory Wallsworth]

Hi,
The title makes this question seem, poor, but highlights what I am trying to do.

I have a large do-file, which calls other do-files. In those other do files, I would like to create temporary files to be used in the larger do-file. tempfile, is cleared at the conclusion of the sub do-files. Below is a brief example
do file 1
-------------------------
use "somefile.dta"
tempfile trial

save "`trial'"
global Trial `trial'
clear
----------------------------

do file 2

do (do file 1)
use "someotherfile.dta"
merge m:1 unique_id using $Trial


I thought the global would be a work around for this, but it seems all the global does is call up the local macro, which after the conclusion of "do file 1" is cleared. Is there a way to use a global to store a file in, and perform actions like this.

As a temporary solution, I can save the file and manually delete it afterwords. But this seems clunky, and I think there must be a way to call a temporary file across do-files.

Thanks for the help.
-Greg

[daniel klein]

Create the tempfiles in your main do-file, then call the sub do-files from the main do-file. A layout could be

/* start main do-file */

[...]

tempfile trail

[...]

do subdo1 "`trail'"

/* end main do-file */

/* start sub do-file subdo1 */

args fn

use "someotherfile.dta"

merge m:1 uniq_id using "`fn'"

/* end sub do-file subdo1*/

Note that I am passing the tempfile's name to the sub do-file, thus no globals are needed.

Hope this helps.

Best
Daniel

[Robert Picard]

Also consider using include instead of do to run your nested code. The temporary files created within the sub do-file would then be visible to the main do-file (as well as within any do-file loaded with include).

[Friedrich Huebler]

You have to use include instead of do. From help include:

include differs from do and run in that any local macros (changed settings, etc.) created by executing the file are not dropped or reset when execution of the file concludes. Rather, results are just as if the commands in filename appeared in the session or file that included filename.

As an example, here is a do-file that we will save as "dofile1.do" and then call from another do-file with include.

sysuse auto, clear
keep mpg
gen id = _n
tempfile trial
save "`trial'"

Here is a second do-file (let's call it "dofile2.do") that executes dofile1.do.

include dofile1.do
sysuse auto, clear
keep make
gen id = _n
merge 1:1 id using "`trial'"
list in 1/5

When we run dofile2.do we get the output below (edited for brevity).

. include dofile1.do
. sysuse auto, clear
. keep mpg
. gen id = _n
. tempfile trial
. save "`trial'"
. sysuse auto, clear
. keep make
. gen id = _n
. merge 1:1 id using "`trial'"

    Result                           # of obs.
    -----------------------------------------
    not matched                             0
    matched                                74  (_merge==3)
    -----------------------------------------

. list in 1/5

     +----------------------------------------+
     | make            id   mpg        _merge |
     |----------------------------------------|
  1. | AMC Concord      1    22   matched (3) |
  2. | AMC Pacer        2    17   matched (3) |
  3. | AMC Spirit       3    22   matched (3) |
  4. | Buick Century    4    20   matched (3) |
  5. | Buick Electra    5    15   matched (3) |
     +----------------------------------------+

As you can see, dofile2.do could access the temporary file created with dofile1.do.

[Maarten Buis]

For an explanation of why Daniels suggestion works, see: http://maartenbuis.nl/publications/temp2sub.html

[Vicky Taxiarchi]

Hello,

Following your conversation in this thread, may I ask for some additional advise? My aim is same as discussed above with the addition of using the parallel command to run the do file faster. Unfortunately the easiest and faster approach include is not a subcommand of parallel. But also parallel does not allow calling a do file with one (or more) argument.

For example:

> tempfile cohort

> parallel setclusters 2

> parallel do subdo cohort

will produce the error:
parallel_normalizepath(): 601 file not found
<istmt>: - function returned error

However, as expected the code

> tempfile cohort

> parallel do subdo cohort

runs absolutely fine.

Any ideas how I could work this out?

Many thanks in advance,
Vicky