Accessing Files Created within Subprogram

[Janet Malzahn]

I am writing a Stata .ado program that performs the same subroutine many times. As such, I would like to turn this subroutine into its own sub-program for efficiency and readability. However, this subroutine involves working with and rewriting multiple intermediate datasets that I now save as tempfiles. From my understanding tempfiles cannot be accessed outside of the program they are created in, so any tempfiles I create in the subroutine would not be accessible in the wider program (and vice-versa).

I know I could just save what I save now as tempfiles as regular datasets (.dta), but since I intend for this command to be used by others I’m not sure if it’s best to save additional (potentially large) files to an arbitrary location. What is the best way to handle this problem? Is there a way to save a dataset to the tempfile directory but not have it delete the file and local path automatically at the conclusion of the program? Is it worth trying to rewrite this subroutine as a program at all? I would like to avoid using Frames so the code can run with slightly earlier versions of stata.

Here is a toy example of the issue at play. I would like to access the intermediate datasets (my_temp1, my_temp2) in the large .ado file the program is called in. The following, as expected, gives an invalid file specification error.

capture program drop my_prog
program my_prog
    sysuse auto
    tempfile my_temp1
    save `my_temp1' // save intermediate tempfiles
    
    sysuse citytemp
    tempfile my_temp2
    save `my_temp2' // save second intermediate file
    end

clear

my_prog

use `my_temp1'

Thanks!

[Clyde Schechter]

You can do this by having my_prog take the temporary filenames as arguments. The temporary filenames are created in the caller and passed to it. These temporary files, then, survive the end of my_prog. Here's a markup of your example:

capture program drop my_prog
program my_prog
    syntax, file1(string) file2(string)
    sysuse auto
    save `file1' // save intermediate tempfiles
    
    sysuse citytemp
    save `file2' // save second intermediate file
    end

clear

tempfile my_temp1 my_temp2

my_prog, file1(`my_temp1') file2(`my_temp2')

use `my_temp1', clear
des

use `my_temp2', clear
des

Whether this is worth doing I can't appraise. Evidently for this toy program it would not be. But if the real program you are dealing with does something useful, and preserving (at least beyond the end of the program and back to the caller) the files is also useful, then, yes, it would be a good idea to do this.

[Nick Cox]

You can pass the name of the temporary file to a subroutine via an option. There are other ways to do it. Here is an example, moderately weird, but it may help.

program my_main
clear 
set obs 10 

tempfile work 
gen foo = 42 
save "`work'"  

my_sub, file("`work'")

end 

program my_sub
syntax , file(string)

use "`file'"
su 

end

[Jared Greathouse]

As a very general rule (for me anyways) subroutines are how you should do all ado-programming. They take a bit of getting used to since you need to know where to put them in the file and how to get the options to work and all that good stuff, but if there's a task you really need to do multiple times, programming is the answer and subroutines usually make all this easier.

[Janet Malzahn]

Awesome, thank you, everyone! I'll take a look into implementing this advice tonight.

[Janet Malzahn]

A quick question: is it going to cause a problem if I name the option the same thing as I call the tempfile I'm modifying in the adofile? So, in your example, Clyde, the options would be called my_temp1 and my_temp2. I think this would be fine since the subroutine program only "knows" the locals passed to it via the options, but I just want to make sure. Thanks!