No announcement yet.
  • Filter
  • Time
  • Show
Clear All
new posts

  • Writing nested foreach loop to convert many .txt files to .dta

    I'm trying to write a foreach loop to convert a few hundred text files, divided up into many folders. So far I have this:

    foreach f in (listed out folders) {
    foreach x in (listed out text files) {
    capture import `x' delimited, delimiter(tab)
    save `x'.dta, replace

    but I keep getting a "no variables defined" r(111) error. I have a feeling it's about the formatting of the list of folders and files, but I've been struggling to figure out the proper way to do so. I apologize in advance if the code is poorly written. I'm an undergraduate in my first semester of learning Stata.

    I didn't want to put the whole lists for the folders and files, because it'd make the post annoying long, but they're listed out in this format:

    foreach f in 03312006 03312007 03312008 etc. and
    foreach x in CI_03312006.txt ENT_03312006.txt etc.

    Thank you.

  • #2
    It's not -import `x' delmited-; you need -import delimited `x'- or -import delimtied using `x'-.

    And even that is not quite right. Since the files are coming from different folders, you need to specify the entire pathname:

    import delimited using `f'/`x', clear delimiter(tab)
    save using `f'/`x', replace
    Because you used -capture-, Stata did not alert you to the fact that your -import delimited- command was failing. And, of course, with -import delimited- failing there was nothing in memory to -save-.

    That is the danger of capture--it allows you to blunder on after an error that really does require your attention. I'm guessing you want to use -capture- because not all combinations of `f' and `x' exist, and you don't want the program to abort when the loop is pointing to non-existent files. So what you need to do is follow the -capture import...- command with a check to verify that the reason -import delimited- failed is because of the anticipated problem of a non-existent file.

    foreach f in list_of_folders {
        foreach x in list_of_filenames {
            capture import delimited using `f'/`x', clear delimiter(tab)
            if c(rc) == 0 { // SUCCESSFUL IMPORT
                save `f'/`x', replace
            else if c(rc) != 601 { // UNANTICIPATED ERROR
                display as error "import delimited failed for unexpected reasons"
                exit c(rc)
            else {
                display as text "file `f'/`x' not found -- skipped"
    This code will enable you to skip over non-existent combinations of `f' and `x' with just a message confirming that this was done, but will cause execution to abort if some other problem arises. When the latter happens, you will also get the error message that you would have gotten without -capture-, so you can start investigating the problem.

    I should also add that based on your example, I am assuming that none of the folder or file names contain embedded spaces. If that assumption is wrong, then references to `f'/`x' need to be surrounded by quotes.


    • #3
      I forgot to reply, but this code ran perfectly and answered every question I had. Thank you very much for the help!