Collapsing with temp files

[Garrett Baker]

Hi, I'm trying to do something like the below, where I collapse to get a sum of the "hours" variable by respondent ID, and then proceed to do a bunch of other data processing (which renders the dataset organized in a way that I can't calculate the "hours" in the correct way) before merging in the "hours" var back in.

The problem that I think is happening is that when I restore the data, the tempfile is not kept in memory--I get the error "invalid file specification" during the merge--but it may be due to something else with how I'm saving the tempfile?

use maindata, clear

preserve
collapse (sum) hours, by(resp_id)
tempfile hoursdata
save `hoursdata'
restore

//
// a bunch of other data cleaning happens 
//

merge 1:m resp_id using `hoursdata'

[Clyde Schechter]

There is nothing obviously wrong with the code.

Here's my hypothesis about what's happening. -tempfile- names are local macros. As such, they go out of scope at the end of whatever "program" defines them. A "program" in this sense includes a block of code highlighted and executed from the do-editor. So I'm guessing that you are running this code line-by-line or in chunks. You can't do that with code that contains local macros. The code must be run without interruption in one fell swoop from the first declaration of hoursdata (in your -tempfile- command) to the last reference to it (in your -merge- command).

If that isn't the solution to your problem, please post back showing example data that replicates the problem. You would also need to show the actual "other data cleaning" code so it can be verified not to be contributing to the problem.

Example data will be necessary for troubleshooting. And to assure that the example data is usable, please use the -dataex- command to show it. If you are running version 18, 17, 16 or a fully updated version 15.1 or 14.2, -dataex- is already part of your official Stata installation. If not, run -ssc install dataex- to get it. Either way, run -help dataex- to read the simple instructions for using it. -dataex- will save you time; it is easier and quicker than typing out tables. It includes complete information about aspects of the data that are often critical to answering your question but cannot be seen from tabular displays or screenshots. It also makes it possible for those who want to help you to create a faithful representation of your example to try out their code, which in turn makes it more likely that their answer will actually work in your data.

[Garrett Baker]

Clyde, your hypothesis was correct. Silly mistake on my part. Thank you.