Using the mv OSX terminal command via Stata's shell tool

[Nicholas Hallman]

Hello all

This is my first post to statalist, so please forgive (but inform me of) and broaches of edict. Also, please note that I am new to Stata on OSX. I just moved from a PC last week.

I am trying to use Stata to append the ".txt" file extension onto a large number of files which are all located in the same directory. I am attempting to use OSX terminal commands to accomplish this (although if there is a Stata command that accomplishes the same thing, that might be preferable). Here is the command I am using in Stata:
! cd "/Users/nicholashallman/Google Drive/Academic/PhD (Mizzou)/Research Projects/Weather/Data/Text"; for file in *; do mv $file "$file.txt"; done

Stata returns the following response:

usage: mv [-f | -i | -n] [-v] source target
mv [-f | -i | -n] [-v] source ... directory
usage: mv [-f | -i | -n] [-v] source target
mv [-f | -i | -n] [-v] source ... directory

What is strange is that if I enter exactly the same command in terminal directly it produces the desired result (namely, all files in the directory "/Users/nicholashallman/Google Drive/Academic/PhD (Mizzou)/Research Projects/Weather/Data/Text" get the .txt extension appended to them). I can't for the life of me figure out why this works when entered directly into terminal but not when entered through Stata. Any help will be greatly appreciated.

[wbuchanan]

Have you tried replacing the first ; with && when calling the line from Stata?

[Nicholas Hallman]

I had not tried replacing the first ; with &&, but I just did and it produces the same result.

[Nicholas Hallman]

I should also mention that the message returned by Stata iterates for each file in the directory. So if there is only one file in the directory I get:

usage: mv [-f | -i | -n] [-v] source target
mv [-f | -i | -n] [-v] source ... directory

If there are two files in the directory I get:

usage: mv [-f | -i | -n] [-v] source target
mv [-f | -i | -n] [-v] source ... directory
usage: mv [-f | -i | -n] [-v] source target
mv [-f | -i | -n] [-v] source ... directory

and so on. So I know that the command is successfully targeting the correct directory. I can also use the "mv" command through Stata without trying to loop it via the "for file in *" command. So if I type:

! mv "/Users/nicholashallman/Google Drive/Academic/PhD (Mizzou)/Research Projects/Weather/Data/Text/010030-99999-2014" "/Users/nicholashallman/Google Drive/Academic/PhD (Mizzou)/Research Projects/Weather/Data/Text/010030-99999-2014.txt"

I get the desired result for the specified file (i.e. 010030-99999-2014). I just need a way to repeat that for many files. I suppose I could do the loop in Stata rather than through the terminal... I suppose that will be what I try next. Even if that works though, I would still like to know what I am doing wrong to cause the behavior described above.

[William Lisowski]

Welcome to Statalist!

In post #1 you give your command as

! cd "/Users/nicholashallman/Google Drive/Academic/PhD (Mizzou)/Research Projects/Weather/Data/Text"; for file in *; do mv $file "$file.txt"; done

Your $file argument lacks the surrounding quotation marks that appear elsewhere, including in the command you show in post #4

! mv "/Users/nicholashallman/Google Drive/Academic/PhD (Mizzou)/Research Projects/Weather/Data/Text/010030-99999-2014" "/Users/nicholashallman/Google Drive/Academic/PhD (Mizzou)/Research Projects/Weather/Data/Text/010030-99999-2014.txt"

Is it possible one of your file names has an embedded space? You seem to have taken care of the spaces in the directory name with your cd prior to the loop.

I keep finding more possibilities and editing this answer. Consider the following, run on OS X 10.10.5, but my account uses tcsh as its default shell.

. ! echo "test $shell"

test 

. ! echo "test \$shell"

test /bin/tcsh

[Alan Riley (StataCorp)]

I am trying to use Stata to append the ".txt" file extension onto a large number of files which are all located in the same directory. I am attempting to use OSX terminal commands to accomplish this (although if there is a Stata command that accomplishes the same thing, that might be preferable). Here is the command I am using in Stata:
! cd "/Users/nicholashallman/Google Drive/Academic/PhD (Mizzou)/Research Projects/Weather/Data/Text"; for file in *; do mv $file "$file.txt"; done

Stata returns the following response:

usage: mv [-f | -i | -n] [-v] source target
mv [-f | -i | -n] [-v] source ... directory
usage: mv [-f | -i | -n] [-v] source target
mv [-f | -i | -n] [-v] source ... directory

What is strange is that if I enter exactly the same command in terminal directly it produces the desired result (namely, all files in the directory "/Users/nicholashallman/Google Drive/Academic/PhD (Mizzou)/Research Projects/Weather/Data/Text" get the .txt extension appended to them). I can't for the life of me figure out why this works when entered directly into terminal but not when entered through Stata. Any help will be greatly appreciated.

The problem is arising because of the $file macro you are using. You are trying to use macros in the shell on OS X. Stata has its own macro syntax, and $ is used to indicate a global macro in Stata.

You are trying to have OS X execute a for loop (as opposed to Stata executing a for loop), and in that for loop, you want OS X to repeatedly fill in the shell macro "file" with each file in a directory. The problem is that before your command line gets passed to OS X, Stata sees you referring to "$file" and tries to expand it in Stata rather than passing the "$file" characters to the shell. Stata says "Oh, the user has referred to $file -- they must want to expand a macro! Do I have a macro named 'file'? No? Ok, I'll substitute an empty string where I see $file." Thus, what OS X ends up getting is this:

! cd "/Users/nicholashallman/Google Drive/Academic/PhD (Mizzou)/Research Projects/Weather/Data/Text"; for file in *; do mv ".txt"; done

and that results in the errors you are getting.

You have two choices:

1) escape the dollar signs in the macro names so that Stata won't try to expand them and will instead pass them on to OS X:

! cd "/Users/nicholashallman/Google Drive/Academic/PhD (Mizzou)/Research Projects/Weather/Data/Text"; for file in *; do mv \$file "\$file.txt"; done

2) perform the cd and loop over the files in Stata, only shelling out to OS X for the mv itself:

cd "/Users/nicholashallman/Google Drive/Academic/PhD (Mizzou)/Research Projects/Weather/Data/Text"
local files : dir "." files "*"
foreach file of local files {
    !mv "`file'" "`file'.txt"
}

Actually, if you are using Stata 14, there is a third way, but it is undocumented so use it at your own risk. Stata 14 has an undocumented command _renamefile. So, you could code an entirely-Stata solution without shelling out to OS X at all:

cd "/Users/nicholashallman/Google Drive/Academic/PhD (Mizzou)/Research Projects/Weather/Data/Text"
local files : dir "." files "*"
foreach file of local files {
    _renamefile "`file'" "`file'.txt"
}