Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Need help changing variable names, and then appending a bunch of files.

    I am trying to change variable names in about 60 files after which I would like to append them and create one large file.
    Each file has the following naming structure avgyyyymm.dta and all of them are stored in the same folder. The files all have exactly the same number of observations and variables (a value for satellite night lights and the latitude and longitude)
    In addition to the existing variables, I would like to add two variables: year (yyyyy) and month (mm) based on the file name.
    Finally, I would like to append all of these files into one master dataset.
    I am getting stuck trying to figure out the correct syntax to save the files with the new variable names in a new folder. The do file below is of two parts. The first part involves changing the variable names and the second part involves appending. I am having problems with with correct way to save the altered files in a new folder in the first part.
    Any help would be appreciated. Thanks.
    *****
    local filelist : dir "G:\resampled" files "avg*.dta"
    foreach f of local filelist {
    use "`f'"
    gen year=substr("`f'",4,4)
    gen month=substr("`f'",8,2)
    save using "G:\resampled_temp\`f'", replace
    }

    cd "G:\resampled_temp"
    ! dir avg*.dta /a-d /b >lightmasterlist.txt
    *create an empty file that will be used to append everything else
    file open myfile using lightmasterlist.txt, read /*reads the first line from the masterlist of files*/
    file read myfile line
    use `line' /*opens the dataset on the first line*/
    save master_lightsdata, replace

    file read myfile line
    while r(eof)==0 { /* while you're not at the end of the file */
    append using `line'
    file read myfile line
    }
    file close myfile
    save master_lightsdata, replace

  • #2
    So there are two problems with your part 1 code. The first is that -save- does not include -using- in its syntax.

    The other problem comes from your use of the backslash (\) as a separator in your pathnames. The difficulty is that the sequence \` is interpreted by Stata as "this ` is not the start of a reference to a local macro--it is a literal ` character." Consequently Stata is not interpreting local macro f when you try to save. The solution I recommend is to use forward slashes (/) as your pathname separator. Even though this is not valid in Windows commands, Stata understands it and makes the necessary change for you automatically when it passes file handling commands to Windows. So, what you need is:

    Code:
    local filelist : dir "G:/resampled" files "avg*.dta"
    foreach f of local filelist {
        use "`f'"
        gen year=substr("`f'",4,4)
        gen month=substr("`f'",8,2)
        save "G:/resampled_temp/`f'", replace
    }
    As for the second part of your code, I don't quite follow what you are trying to do here, and I don't see that you are asking any questions about it any way. But I can't help thinking that you will get the same functionality more simply by using Robert Picard's -filelist- program, available from SSC.

    Comment


    • #3
      Clyde
      Thanks for your suggestions. Those were very elementary mistakes!
      Everything worked very well.
      In the second part, I was trying to append all the files (which also went through smoothly). I got the code here
      https://stats.idre.ucla.edu/stata/fa...mber-of-files/

      I will check out the filelist program too.

      Comment

      Working...
      X