global temporary dataset for interactive use in multiple DO file

[Ataullah Khan]

I intend to use the `temp' file in multiple DO files, however, the file vanishes as the first do file concludes.

I tried the following commands and variation of it, but all in vain.

tempfile main
save `main',emptyok
global main `main'

use $main
Do1.do
clear

use $mian
Do2.do
clear

Can someone help me out on how to save the temporary file in a global macro rather than a local.
Thanks

[Joseph Coveney]

Pass it as an argument to the do-files.

*! master.do
tempfile main
save `main', emptyok

do do1 `main'
do do2 `main'

exit

*! do1.do

local main `0'

use `main', clear

exit

*! do2.do

local main `0'

use `main', clear

exit

[Ataullah Khan]

Hi, sorry I did not understood your answer.

I want to save the results from all do file into a one master file named as main.
I wrote the following code, but `main' is deleted as do1 finishes.

clear all
macro drop _all

tempfile main
save `main'

do "do1.do"
append using `$main'
clear

do "do2.do"
append using `$main'
clear

do "do2.do"
append using `$main'
clear

use `main',clear

Thanks

[William Lisowski]

Without knowing what your code in do1.do is like it is difficult to know what you hope to have happen. Is it your plan that do1.do will write a dataset that then will be appended to the tempfile in the calling do-file? If so, then this variation on Joseph Coveney's code may point you in a useful direction.

// this is do1.do, all the others will be similar
local output `"`1'"`
...
save `"`output'"', replace
...

// this is the main do-file
clear all
macro drop _all

// create an empty dataset to combine the results
tempfile main
save `"`main'"'

// create a tempfile for the do-files to place their results in
tempfile results

// run do1 and append the results to the main dataset
do do1 `"`results'"'
use `"`main'"', clear
append using `"`results'"'
save `"`main'"', replace
clear

// run do2 and append the results to the main dataset
do do2 `"`results'"'
use `"`main'"', clear
append using `"`results'"'
save `"`main'"', replace
clear

// read the combined results into memory
use `"`main'"', clear
...

Note that I surround the temporary filenames with compound double quotes

`" and "'

to avoid problems with spaces and special characters in the file path.

[Ataullah Khan]

The code in do1.do generate data and perform some isolated calculations, which then I intend to combined in a main data set.
I tried your code but it issued error.

cd "C:\path name\abc"

clear all
macro drop _all

tempfile main
save `main',emptyok

// create a tempfile for the do-files to place their results in
tempfile results

// attendance calculation and append the results to the main dataset
do "attendance.do" `results'
use `main', clear
append using `results'

error r(601) occurs at this stage.
message: file Temp\ST_01000002.tmp not found

and end of do file

I am using single quotes since there are no spaces and special letter in filenames.

[William Lisowski]

Here is a working demonstration of my code. I suggest you compare your code to it and concentrate on the areas where there are differences.

// do1.do
local output `"`1'"'
clear
set obs 5
generate x = 1
save `"`output'"', replace

// do2.do
local output `"`1'"'
clear
set obs 5
generate x = 2
save `"`output'"', replace

clear all
macro drop _all

// create an empty dataset to combine the results
tempfile main
save `"`main'"', emptyok

// create a tempfile for the do-files to place their results in
tempfile results

// run do1 and append the results to the main dataset
do do1 `"`results'"'
use `"`main'"', clear
append using `"`results'"'
save `"`main'"', replace
clear

// run do2 and append the results to the main dataset
do do2 `"`results'"'
use `"`main'"', clear
append using `"`results'"'
save `"`main'"', replace
clear

// read the combined results into memory
use `"`main'"', clear
list, noobs

. clear all

. macro drop _all

. 
. // create an empty dataset to combine the results
. tempfile main

. save `"`main'"', emptyok
(note: dataset contains 0 observations)
file /var/folders/xr/lm5ccr996k7dspxs35yqzyt80000gp/T//S_17438.000001 saved

. 
. // create a tempfile for the do-files to place their results in
. tempfile results

. 
. // run do1 and append the results to the main dataset
. do do1 `"`results'"'

. // do1.do
. local output `"`1'"'

. clear

. set obs 5
number of observations (_N) was 0, now 5

. generate x = 1

. save `"`output'"', replace
(note: file /var/folders/xr/lm5ccr996k7dspxs35yqzyt80000gp/T//S_17438.000002 not found)
file /var/folders/xr/lm5ccr996k7dspxs35yqzyt80000gp/T//S_17438.000002 saved

. 
end of do-file

. use `"`main'"', clear

. append using `"`results'"'

. save `"`main'"', replace
file /var/folders/xr/lm5ccr996k7dspxs35yqzyt80000gp/T//S_17438.000001 saved

. clear

. 
. // run do2 and append the results to the main dataset
. do do2 `"`results'"'

. // do2.do
. local output `"`1'"'

. clear

. set obs 5
number of observations (_N) was 0, now 5

. generate x = 2

. save `"`output'"', replace
file /var/folders/xr/lm5ccr996k7dspxs35yqzyt80000gp/T//S_17438.000002 saved

. 
end of do-file

. use `"`main'"', clear

. append using `"`results'"'

. save `"`main'"', replace
file /var/folders/xr/lm5ccr996k7dspxs35yqzyt80000gp/T//S_17438.000001 saved

. clear

. 
. // read the combined results into memory
. use `"`main'"', clear

. list, noobs

  +---+
  | x |
  |---|
  | 1 |
  | 1 |
  | 1 |
  | 1 |
  | 1 |
  |---|
  | 2 |
  | 2 |
  | 2 |
  | 2 |
  | 2 |
  +---+

. 
end of do-file

[William Lisowski]

Let me add one more thing that I have thought of. This may not have any bearing on your problem, but in case it does, here it is.

It is required that you run your entire main do-file at once.

Consider the following example. In the do-file editor window, I have a two-line program that I run in its entirety.

. do "/Users/lisowskiw/Downloads/example.do"

. local message Hello, world.

. display "`message'"
Hello, world.

. 
end of do-file

Now I run the same two lines by selecting the first line and running it, then selecting the second line and running it.

. do "/var/folders/xr/lm5ccr996k7dspxs35yqzyt80000gp/T//SD30770.000000"

. local message Hello, world.

.
end of do-file

. do "/var/folders/xr/lm5ccr996k7dspxs35yqzyt80000gp/T//SD30770.000000"

. display "`message'"


.
end of do-file

The important thing to keep in mind is that local macros vanish when the do-file within which they were created ends. If you look carefully at the results above, you'll see that when I selected a single line to run, it was copied into a temporary do-file and run, so even though both lines are in the same window in the do-file editor, they are run as separate do-files, and local macro defined in the first line vanishes at the end of that do-file, and is undefined when the second line is run.

[Ataullah Khan]

Thank you William Lisowski. The code above worked perfect with some minor tweaks.