No announcement yet.
  • Filter
  • Time
  • Show
Clear All
new posts

  • combine csv files

    Hi all,
    I have 1460 file.csv, each one of these files provide information on price, brand and fuel of different petrol stations.
    Each of this file represents a day of the year.

    My question is: can I append this file directly without turning them into dta?

    Thanks for your help and sorry for my unclear English.

  • #2
    Naturally you can concatenate them using your operating system, a scripting language, etc. Questions include do they contain header lines? are they exactly compatible? Why would you want to do that?


    • #3
      Thanks Nick,
      Yes, all files have the same number of variables and same name as well, so they are exactly compatible; Do you mean something different saying "do they contain header lines"?
      My need is to create a unique dataset with 1460 days (4 years), then I need to automatize that process.


      • #4
        I mean do they contain header lines, such as

        price brand fuel


        • #5
          for example, the structure of these files is:

          idstation price brand fuel time address city
          1258 1.658 Q8 diesel 2012/01/01 streat oasis 58 londra
          ..... ..... ..... ..... ..... ..... .....
          ..... ..... ..... ..... ..... ..... .....
          ..... ..... ..... ..... ..... ..... .....
          ..... ..... ..... ..... ..... ..... .....

          all files have these features


          • #6
            I doubt there is any useful shortcut over reading them into Stata and appending. Your string and date variables may need work.


            • #7
              Thanks for your answers Nick


              • #8
                This could be done with Stata, but I don't see an efficient way around it other than doing as Nick suggests, and reading in each file one at a time. If the files have precisely the same structure, a shell script on Linux/Mac is probably more efficient, and then import into Stata for data management.

                To give a hint on the shell script, you may find this useful:
                tail --lines=+2 $filename > $newfile 
                This will print out an entire file, skipping the first line, which you may redirect to a new file.
                Last edited by Leonardo Guizzetti; 12 Feb 2019, 08:08.