Allow Stata to continue with do file after invoking shell

[Nicholas Hallman]

I am attempting to write a do file that launches several subsidiary do files in the background using shell in MacOS. I can write:

!export PATH=$PATH:/Applications/Stata/StataSE.app/Contents/MacOS/ ; StataSE -b do Sub1.do
!export PATH=$PATH:/Applications/Stata/StataSE.app/Contents/MacOS/ ; StataSE -b do Sub2.do

which runs the Sub1.do file in the background as intended. The problem is that Stata waits for shell to complete Sub1.do before moving on with the parent do file and initiating Sub2.do. I would like Stata to instead move on to immediately initiate Sub2.do so that both subsidiary do files can run simultaneously in the background. Is this possible? Thanks in advance.

[William Lisowski]

The output of help shell tells us

shell (synonym: "!") allows you to send commands to your operating system or to enter your
operating system for interactive use. Stata will wait for the shell to close or the
operating_system_command to complete before continuing.

so what you seek does not seem possible.

Section B.3 "Running Stata in batch mode" in the Getting Started With Stata For Mac PDF (included with your Stata installation and accessible through Stata's Help menu) suggests that it is possible to run multiple Stata commands in batch in the background when started from the terminal. Perhaps this provides a direction forward.

[Nicholas Hallman]

Yes - what you suggest is my back-up plan. I would like to avoid manually starting each batch process at the terminal window and was hoping to automate it with a parent do file but, as you say, this may not be possible. I was hoping that perhaps someone here knew of some workaround.

[William Lisowski]

On macOS the following submits a shell script containing two commands that run in parallel. However, the shell running the script does not exit until all the background processes have completed. Not sure if this would meet your needs.

. type commands
echo "one" ; sleep 5 &
echo "two" ; sleep 5 &
ps -u lisowskiw | grep sleep
echo "exiting"

. ! sh < commands

one
two
  502  2208 ??         0:00.00 sleep 5
  502  2209 ??         0:00.00 sleep 5
  502  2211 ??         0:00.00 grep sleep
exiting

.

[Nicholas Hallman]

Interesting - I will admit that I am a little out of my depth messing with shell, so it will take me some time to play around and figure out exactly what you have done here. But this does seem to meet my need. I will post again if for some reason this doesn't end up working for me. Thanks so much!

[Daniel Feenberg]

Are we sure that just adding a "&" at the end of the command won't do the trick? I just tried opening Stata 16 and typing:

!ping  host.foo &

and got a stata prompt while the ping continued to respond. I tested on Linux, so this is not conclusive, but it is suggestive.
OSx apparently does not have "batch" or "at" commands which would be alternatives in most Unix-like systems.

[Nicholas Hallman]

No dice on MacOS (unless I am overlooking something). When I type:

!ping google.com &
display "works!"

Stata begins pinging google.com and outputting the results to the Stata results window. The text "works!" is never displayed, and I have to force quit Stata to regain control. Thanks for the attempt, though!

[William Lisowski]

OSx apparently does not have "batch" or "at" commands which would be alternatives in most Unix-like systems.

In theory nothing prevents Stata for Mac from behaving as Stata for linux. To be clear, macOS (no longer OS X) is a full Unix-like system, and has the at and batch commands. Submitting a file of commands using the at command perhaps would be an approach to take. However, the output of the at command is emailed to the user (at least according to the man page I am looking at) and in general macOS systems are not set up with sendmail daemons running.

The implementation of the Stata shell command in Stata for Mac apparently chooses not to return control to Stata until the shell process and all the processes it spawns have completed, as demonstrated in post #4 above. Since any output to stdout or stderr from the processes by default returns to Stata's Results window, I'm not sure how this could be otherwise. I can't imagine how your test would have looked had you run a Stata command from your Command window while the ping was displaying its output in the same Results window to be used by your Stata command.

In this and the post above at #4 I have been implicitly assuming the background commands were run from an interactive windowed Stata session.

[Mustafa Ugur Karakaplan]

I had the same issue, and I searched and found this post unresolved. Then I found a solution, so I wanted to share it with everybody who search for the same issue and find this post: Use winexec instead of shell. winexec works in Windows, Mac, and Unix GUI, and it works like shell except that Stata will continue without waiting for the program run with winexec to complete.

winexec ping google.com
di "didn't wait!"

This code displays the message without waiting for the ping.

help winexec

for more information.

[Tiago Pereira]

Not sure if this is going to work in Mac, but it is routine in linux. As pointed out above, something like this should work:

for i in {1..1..1}; do
 echo "Number: $i"
          /usr/local/stata16/xstata-se do "/home/xxx/Desktop/script1.do"  & sleep 2s
          /usr/local/stata16/xstata-se do "/home/xxx/Desktop/script2.do"  & sleep 2s
   done

The scripts will run in parallel and set 2 seconds apart.