Loop opening txt file and create a variable

[Marco Errico]

Hi all,

I have a series of txt file stored in a folder.

I would love to do the following (ideally via loop):
1-Open each file
2-Create a variable with same name as the file title (so it will identify the file)
3-Stored all the files in dta within the folder.

Finally, I would love to append all dta file so to have a one single dataset.

Many thanks for any advice you might have!

[Clyde Schechter]

On the assumption that the files are all and only the files with filename extension .txt, and that they are delimited (not fixed-width) files:

local filelist: dir "." files "*.txt"

foreach f of local filelist {
    import delimited using "`f'", clear varnames(1)
    gen source = "`f'"
    local dta_name: subinstr local f ".txt" ".dta"
    save `dta_name', replace    
}

will accomplish 1, 2, and 3. You must have the directory containing the text files as the working directory when you run this--otherwise Stata will see it can't find the files. The code assumes that the first row in each text file contains the names of the variables.

As for appending them all together, that is a bad idea. There is a high probability that there will be incompatibilities among the files, so that the appended result will be an unusable mess. So, I suggest getting Mark Chatfield's -precombine- program from SSC and using that to identify any such problems first. Then go and fix them in the files separately, and verify that -precombine- finds no problems in the fixed files. Once that is done, it is simple to append them all.

[Marco Errico]

Hi Clyde Schechter, thanks for you kind replying.

I took advantage of the code that you suggested

local filelist: dir "C:\Users\M.ERRICO\Desktop\IMU_work\Municipal_Elections\2004_2016" files "*.txt"

foreach f of local filelist {
    import delimited using "`f'", clear varnames(1)
    gen date = "`f'"
    local dta_name: subinstr local f ".txt" ".dta"
    save `dta_name', replace    
}

But I get the following error message

-file comunali-20040626.txt not found-

Which is a bit weird to me as the first file in the folder is indeed "comunali-20040626.txt"

Do you have any suggestions?


Edit: It seems I fixed by introducing the following

 local filelist: dir "C:\Users\M.ERRICO\Desktop\IMU_work\Municipal_Elections\2004_2016"files "*.txt"
cd "C:\Users\M.ERRICO\Desktop\IMU_work\Municipal_Elections\2004_2016"
foreach f of local filelist{    
import delimited using "`f'", clear varnames(1)    
gen source = "`f'"    
local dta_name: subinstr local f ".txt" ".dta"  
save `dta_name', replace    
}

Many thanks.
Marco

[Clyde Schechter]

Edit: It seems I fixed by introducing the following
Code:

local filelist: dir "C:\Users\M.ERRICO\Desktop\IMU_work\Municipal_Elec tions\2004_2016"files "*.txt" cd "C:\Users\M.ERRICO\Desktop\IMU_work\Municipal_Elec tions\2004_2016"

As I said in #2:

You must have the directory containing the text files as the working directory when you run this--otherwise Stata will see it can't find the files.

[Marco Errico]

Indeed, you were right. Many thanks Clyde Schechter !