file ___________ not found

[Mantasha Husain]

Hi everyone,

I have written this code to import data into Stata (by already mentioning multiple datasets earlier):

foreach i of local surveybatch {
insheet using "$rawdata\survey_data\\`i'_e-survey.csv", comma names clear
keep v281 phone operator patientid survname calc_phone_number submissiondate postpaid
local k: subinstr local i "." "", all
tempfile `k'
save `k', replace
disp `k'
}



local surveybatch is already defined in the .do file. When it starts importing all the active datasets mentioned, Stata tells me that the file was not found. It just stops at the first dataset (even though when I open the Data Editor window, I see only the first dataset imported). Even when I reduce the number of datasets, it will always tell me that the first dataset was not found (even though that's the only dataset visible in the Data Editor). It doesn't move on to the other datasets at all. It tells me :

(note: file 20170901.dta not found)
file 20170901.dta could not be opened




Any one got any idea why this is happening? I have already checked my paths and made sure the files exist.

Really tired of trying to figure out the problem!

[Nick Cox]

I avoid backslashes in this kind of thing because I have read (and indeed written) http://www.stata-journal.com/sjpdf.h...iclenum=pr0042

Your use of double backslashes implies that you are aware of this problem, but even so forward slashes are easier.

I would rewrite this as

foreach i of local surveybatch {
    insheet using "$rawdata/survey_data/`i'_e-survey.csv", comma names clear

    keep v281 phone operator patientid survname calc_phone_number ///
    submissiondate postpaid

    local k: subinstr local i "." "", all  
    disp "`k'"
    save `k', replace
}

Here I note in addition to using forward slashes that

1. The quotation marks in the display command are essential for what you want.

2. The tempfile command seems likely to be based on a confusion to me, so I cut it out. Conversely, if you're sure it's right, you may need to explain why.

If this doesn't help, you may need to show the contents of surveybatch and rawdata because we can't see them from where we sit.

EDIT: Clyde's analysis below goes deeper.

[Clyde Schechter]

The file not found you are getting is coming from -save `k', replace- and is not a problem. `k' is a file that does not yet exist. The -save- command, seeing the -replace- option, is simply notifying you that there is nothing there to replace. But that doesn't do you any harm. It's just an informational message. Where you are hitting trouble is with the -file 20170901.dta could not be opened- message.

I imagine that you think you are saving the data to the -tempfile `k'- that you declared. But you are not. YOu are asking Stata to save to a permanent file whose name is the content of the current value of local macro k; in this instance you are trying to create a new permanent data file 20170901.dta. My guess is that the reason that is failing is that you do not have write permissions for the directory you are working in. But if I am correct that you don't want a permanent file, but a tempfile, then you need not worry about write permissions here, because tempfiles are saved in a different directory. So you need to fix your code so that it actually saves to the tempfile `k'. To do that:

save ``k'', replace

Note: unless local macro survey batch contains names that, when stripped of . characters, are not distinct, you do not need the -replace- option, because none of these files will be pre-existing.

Added: Crossed with #2. If my guess that Mantasha does not have write privileges in the working directory is correct, then Nick's solution will not work, for the same reason. Our solutions are similar in that both of us recognize that the sequence

tempfile `k'
save `k'

does not save a tempfile (and in most situation actually makes no sense). His response was to remove the -tempfile- statement, and mine is to correct the -save- command so it corresponds to the -tempfile-.