Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • importing, then appending, 1451 files


    I am trying to import and then append 1451 csv files
    Below is my do file:
    Code:
      
     local filelist: dir "Macintosh HD/Users/rich/projects/ingrid parry/data/csv/csv_data/" files "*.csv"  
     foreach f of local filelist {  
        import delimited "Macintosh HD/Users/rich/projects/ingrid parry/data/csv/csv_data`f'", varn(1) stringc(_all)  
        local f = subinstr("`f'", "csv", ".dta",1)  
        save ""/Users/rich/projects/ingrid parry/data/csv/csv_data/`f'", replace  }
    added: the "}" is really on a different line - it just posted incorrect

    Note that I tried with and without "Macintosh HD" and with and without a slash at the end of the first quoted material; before writing the above, I tried:
    Code:
     pwd  
     /Users/rich/projects/ingrid parry/data/csv/csv_data
    which is what I first put into my "local" command in the do file. However, when I run this, I get the following error:
    Code:
      
     directory Macintosh HD/Users/rich/projects/ingrid parry/data/csv/csv_data not found
    when I include "Macintosh HD", and
    Code:
      
     file /Users/rich/projects/ingrid parry/data/csv/csv_dataCIN0456.csv not found
    when I do not include "Macintosh HD"; in each case, the return code is 601
    as a second question, I want to then append all 1,451 files into one file - what is the most efficient way to do that please

  • #2
    A clue to the answer lies in that last error message. Surely there needs to be a slash after "csv_data" to separate it from "CIN0456". Was there a slash at the end of the directory path when you tested the version without "Macintosh HD"? I also notice that you are modifying local macro "f" inside the forvalues loop. I don't think that will be allowed.

    Comment


    • #3
      thanks but I get the same message with or without the slash

      forgot to mention that I based my code on a post by Hemanshu Kumar at https://www.statalist.org/forums/for...as-stata-files

      Comment


      • #4
        Try:

        Code:
        import delimited "/Users/rich/projects/ingrid parry/data/csv/csv_data/`f'", varn(1) stringc(_all)
        Notice the slash added after "csv_data". I do think that is the problem.

        Comment


        • #5
          just to close this out, the following was supplied by tech support and worked:
          Code:
          change directories into the folder containing all the csv files
          first, then you can run the loop over the files:
          
          cd "C:\Data\csvfiles" (or whatever path you're using- you can also
          use the File > Change working directory... menu in Stata to Browse
          to the folder)
          
          clear
          local csvfiles : dir . files "*.csv"
          
          foreach file of local csvfiles {
          import delimited using `"`file'"'
          
          gen from = `"`file'"'
          
          capture append using `"`master'"'
          
          tempfile master
          save `"`master'"', replace
          clear
          }
          
          use `"`master'"', clear

          Comment

          Working...
          X