Collecting names of temp files in a local

[Pablo Mitnik]

Hi all,

I need to collect in a local the names of all temp files created by a do file at some point of its execution. By "names" I mean the real names used by Stata to save to disk, including the full path.

I can see what these names are by using - macro dir - or - macro list - . But although I believe I should be able to collect the names with the existing macro functions, I have not been able to figure out how.

Any help would be greatly appreciated.

Pablo

[Leonardo Guizzetti]

There isn't a convenience function for this. The reason I can think of why this might be a bad idea to implement as a function is that macros are mutable, and can change at anytime (when you code it, that is).

A second, unanswered, concern is why you might think you want or need this? By definition, Stata will tidy up these files for you at the end of execution. Perhaps you don't want the additional disk clutter, that may be reasonable, or there may be better workarounds.

The simplest approach is is to keep one more local macro, call it your file list. Then store the name (not the expanded contents) of each new tmpfile name. Loop through them later to perform some action.

[Jared Greathouse]

Yeah I can imagine situations where you wanna append tempfiles and all that good stuff, but I don't know when you'd went to use them for other purposes

[Leonardo Guizzetti]

Yeah I can imagine situations where you wanna append tempfiles and all that good stuff, but I don't know when you'd went to use them for other purposes

Such a workaround in this case would be to use frames instead, then drop the frames. It keeps your hands clean of the tempfiles.

[Pablo Mitnik]

Thanks much for your answers.

The reason I asked this question is that I am working with really big data in a context in which space constrains are binding. Unfortunately, in the server I am working, Stata is not deleting the tempfiles automatically. Stata technical support has been contacted, but we haven't received a fix yet. So, in the meantime, I am writing a program that will address the problems this causes.

There are several ways in which I can collect the needed information as the do file is executed and save it to a txt file (saving to disk is required because I need to be able to delete tempfiles even if, e.g., the job is killed.); that info will then be used by the program I am writing to selectively delete the tempfiles left behind by the do file in question, not all temp files (i.e., I don't want to delete tempfiles created by other do or ado files). But, programming-wise, those forms of collecting information are less convenient than if a macro function had been available to do it in one line.

[Leonardo Guizzetti]

Stata should have appropriate write permissions to be able to create and delete tmp files from your temporary directory. Since you can create them, you can also delete them, but I have noticed that on occasion these aren't always changed up. However, this only happens at the end of the execution of your do file which may be inconvenient.

One idea that comes to mind, is to create permanent datasets that you can then later clean up when you like. For appending large files together, there is user-contributed command called -chunky- I believe, that you can investigate.

[Pablo Mitnik]

For appending large files together, there is user-contributed command called -chunky- I believe, that you can investigate.

Thanks! I will take a look.