Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Local macro to import multiple Excel files

    I am trying to compile data from a series of Excel workbooks and worksheets, while I have something that works it is rather inefficient coding. I have got company data from 30 companies in Excel format, and these Excel workbooks are in (relatively) consistent structure. The approach I've taken is to simply list each company every time I need to use that foreach loop. For example:
    Code:
     
    foreach company in "Company Alpha" "Company Beta" "Company Gamma"  {
      import excel "`company'", sheet("sheetname") clear
      *code applicable to each company
      }
    However rather than listing all 30 company names every time, I thought I could apply a local macro, but regardless of how I structure the local macro I cannot seem to make it work.
    Code:
    local names `" "Company Alpha" "Company Beta" "Company Gamma" "'
    foreach company in `names' {
       import excel "`company'", sheet("sheetname") clear
      *code applicable to each company
      }
    This does nothing. I tried adding extra " " in the loop, but get an r(100) using error. Any advice on what I am doing wrong, or if it can be done. Any assistance would be much appreciated

    PS - I did consider using a local files macro, and had it working to an extent, but because I also use the company loop names to define variables and save data file makes this method not ideal as it loops the full file name rather than the company name
    Code:
    local files : dir "C:\Users\MichaelW\Desktop\Stata working drive" files "*.xlsx"
    foreach company in `files' {
     *code applicable to each company
      generate company="`company'"
      generate year="`year'"  
      save "C:\Users\MichaelW\Desktop\Stata working drive" files\_`company'.dta", replace
    }

  • #2
    Instead of
    Code:
    foreach file in `file_list' {
    , I've tended to use
    Code:
    foreach file of local file_list {
    Ditto for the local you get from levelsof.

    Anyway, you can try something like that below (start after the "Begin here" comment; the first part is just to create some toy Excel workbooks for illustration).

    It illustrates how to loop through locals of items that contain spaces, how to get the company name from the file name, doing the code applicable to each company and the rest.

    Substitute your directory names in for the "." (current working directory) that I use in the illustration.

    .ÿversionÿ14.0

    .ÿ
    .ÿclearÿ*

    .ÿsetÿmoreÿoff

    .ÿsetÿseedÿ`=date("2015-08-27",ÿ"YMD")'

    .ÿ
    .ÿquietlyÿsetÿobsÿ1

    .ÿquietlyÿgenerateÿstrÿgreetingsÿ=ÿ""

    .ÿquietlyÿgenerateÿintÿranduÿ=ÿ.

    .ÿ
    .ÿforeachÿletterÿinÿAÿBÿCÿ{
    ÿÿ2.ÿÿÿÿÿÿÿÿÿquietlyÿreplaceÿgreetingsÿ=ÿ"Hi!ÿI'mÿCompanyÿ`letter'!"
    ÿÿ3.ÿÿÿÿÿÿÿÿÿquietlyÿreplaceÿranduÿ=ÿfloor((1999ÿ-ÿ1972ÿ+ÿ1)ÿ*ÿruniform()ÿ+ÿ1972)
    ÿÿ4.ÿÿÿÿÿÿÿÿÿquietlyÿexportÿexcelÿ"Companyÿ`letter'.xlsx",ÿsheet(Whatever)ÿfirstrow(variables)ÿreplace
    ÿÿ5.ÿ}

    .ÿ
    .ÿ*
    .ÿ*ÿBeginÿhere
    .ÿ*
    .ÿ
    .ÿlocalÿexcel_filesÿ:ÿdirÿ"."ÿfilesÿ"*.xlsx"

    .ÿ
    .ÿforeachÿexcel_fileÿofÿlocalÿexcel_filesÿ{
    ÿÿ2.ÿ
    .ÿÿÿÿÿÿÿÿÿimportÿexcelÿ"`excel_file'",ÿsheet(Whatever)ÿfirstrowÿclear
    ÿÿ3.ÿ
    .ÿÿÿÿÿÿÿÿÿlocalÿcompanyÿ:ÿsubinstrÿlocalÿexcel_fileÿ".xlsx"ÿ""
    ÿÿ4.ÿÿÿÿÿÿÿÿÿgenerateÿstrÿcompanyÿ=ÿ"`company'"
    ÿÿ5.ÿ
    .ÿÿÿÿÿÿÿÿÿlocalÿyearÿ=ÿrandu[1]
    ÿÿ6.ÿÿÿÿÿÿÿÿÿgenerateÿstrÿyearÿ=ÿ"`year'"
    ÿÿ7.ÿ
    .ÿÿÿÿÿÿÿÿÿquietlyÿsaveÿ"`company'",ÿreplace
    ÿÿ8.ÿÿÿÿÿÿÿÿÿlist,ÿnoobsÿabbreviate(20)
    ÿÿ9.ÿ}

    ÿÿ+-----------------------------------------------+
    ÿÿ|ÿÿÿÿÿÿÿÿÿÿgreetingsÿÿÿranduÿÿÿÿÿcompanyÿÿÿyearÿ|
    ÿÿ|-----------------------------------------------|
    ÿÿ|ÿHi!ÿI'mÿCompanyÿA!ÿÿÿÿ1973ÿÿÿCompanyÿAÿÿÿ1973ÿ|
    ÿÿ+-----------------------------------------------+

    ÿÿ+-----------------------------------------------+
    ÿÿ|ÿÿÿÿÿÿÿÿÿÿgreetingsÿÿÿranduÿÿÿÿÿcompanyÿÿÿyearÿ|
    ÿÿ|-----------------------------------------------|
    ÿÿ|ÿHi!ÿI'mÿCompanyÿB!ÿÿÿÿ1993ÿÿÿCompanyÿBÿÿÿ1993ÿ|
    ÿÿ+-----------------------------------------------+

    ÿÿ+-----------------------------------------------+
    ÿÿ|ÿÿÿÿÿÿÿÿÿÿgreetingsÿÿÿranduÿÿÿÿÿcompanyÿÿÿyearÿ|
    ÿÿ|-----------------------------------------------|
    ÿÿ|ÿHi!ÿI'mÿCompanyÿC!ÿÿÿÿ1982ÿÿÿCompanyÿCÿÿÿ1982ÿ|
    ÿÿ+-----------------------------------------------+

    .ÿ
    .ÿlocalÿstata_filesÿ:ÿdirÿ"."ÿfilesÿ"*.dta"

    .ÿlocalÿline_sizeÿ`c(linesize)'

    .ÿsetÿlinesizeÿ80

    .ÿforeachÿdatasetÿofÿlocalÿstata_filesÿ{
    ÿÿ2.ÿÿÿÿÿÿÿÿÿuseÿ"`dataset'",ÿclear
    ÿÿ3.ÿÿÿÿÿÿÿÿÿdescribe
    ÿÿ4.ÿ}

    ContainsÿdataÿfromÿCompanyÿA.dta
    ÿÿobs:ÿÿÿÿÿÿÿÿÿÿÿÿÿ1ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
    ÿvars:ÿÿÿÿÿÿÿÿÿÿÿÿÿ4ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ27ÿAugÿ2015ÿ13:45
    ÿsize:ÿÿÿÿÿÿÿÿÿÿÿÿ33ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
    --------------------------------------------------------------------------------
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿstorageÿÿÿdisplayÿÿÿÿvalue
    variableÿnameÿÿÿtypeÿÿÿÿformatÿÿÿÿÿlabelÿÿÿÿÿÿvariableÿlabel
    --------------------------------------------------------------------------------
    greetingsÿÿÿÿÿÿÿstr18ÿÿÿ%18sÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿgreetings
    randuÿÿÿÿÿÿÿÿÿÿÿintÿÿÿÿÿ%10.0gÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿrandu
    companyÿÿÿÿÿÿÿÿÿstr9ÿÿÿÿ%9sÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
    yearÿÿÿÿÿÿÿÿÿÿÿÿstr4ÿÿÿÿ%9sÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
    --------------------------------------------------------------------------------
    Sortedÿby:ÿ

    ContainsÿdataÿfromÿCompanyÿB.dta
    ÿÿobs:ÿÿÿÿÿÿÿÿÿÿÿÿÿ1ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
    ÿvars:ÿÿÿÿÿÿÿÿÿÿÿÿÿ4ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ27ÿAugÿ2015ÿ13:45
    ÿsize:ÿÿÿÿÿÿÿÿÿÿÿÿ33ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
    --------------------------------------------------------------------------------
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿstorageÿÿÿdisplayÿÿÿÿvalue
    variableÿnameÿÿÿtypeÿÿÿÿformatÿÿÿÿÿlabelÿÿÿÿÿÿvariableÿlabel
    --------------------------------------------------------------------------------
    greetingsÿÿÿÿÿÿÿstr18ÿÿÿ%18sÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿgreetings
    randuÿÿÿÿÿÿÿÿÿÿÿintÿÿÿÿÿ%10.0gÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿrandu
    companyÿÿÿÿÿÿÿÿÿstr9ÿÿÿÿ%9sÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
    yearÿÿÿÿÿÿÿÿÿÿÿÿstr4ÿÿÿÿ%9sÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
    --------------------------------------------------------------------------------
    Sortedÿby:ÿ

    ContainsÿdataÿfromÿCompanyÿC.dta
    ÿÿobs:ÿÿÿÿÿÿÿÿÿÿÿÿÿ1ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
    ÿvars:ÿÿÿÿÿÿÿÿÿÿÿÿÿ4ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ27ÿAugÿ2015ÿ13:45
    ÿsize:ÿÿÿÿÿÿÿÿÿÿÿÿ33ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
    --------------------------------------------------------------------------------
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿstorageÿÿÿdisplayÿÿÿÿvalue
    variableÿnameÿÿÿtypeÿÿÿÿformatÿÿÿÿÿlabelÿÿÿÿÿÿvariableÿlabel
    --------------------------------------------------------------------------------
    greetingsÿÿÿÿÿÿÿstr18ÿÿÿ%18sÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿgreetings
    randuÿÿÿÿÿÿÿÿÿÿÿintÿÿÿÿÿ%10.0gÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿrandu
    companyÿÿÿÿÿÿÿÿÿstr9ÿÿÿÿ%9sÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
    yearÿÿÿÿÿÿÿÿÿÿÿÿstr4ÿÿÿÿ%9sÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
    --------------------------------------------------------------------------------
    Sortedÿby:ÿ

    .ÿsetÿlinesizeÿ`line_size'

    .ÿ
    .ÿexit

    endÿofÿdo-file


    .


    Comment


    • #3
      For related tasks also see xls2dta (SSC).

      Best
      Daniel

      Comment


      • #4
        Thanks Joseph for your response, the subinstr macro was really useful.

        Just one quick question, when I use this code, I find that my company names (as derived from the excel file names) are all converted to lower case, although I see in Joseph's code they convert the case over consistent with the file name. Could this be a versioning issue (We have version 13.0). No biggie as workaround is easy enough, but just curious.

        Thanks
        Mike

        Comment


        • #5
          Originally posted by Michael Wallace View Post
          Just one quick question, when I use this code, I find that my company names (as derived from the excel file names) are all converted to lower case, although I see in Joseph's code they convert the case over consistent with the file name. Could this be a versioning issue (We have version 13.0).
          Instead of
          Code:
          local excel_files : dir "." files "*.xlsx"
          type
          Code:
          local excel_files : dir "." files "*.xlsx", respectcase
          It's a problem with Microsoft Windows and not with Stata versions. See help extended_fcn.

          Comment


          • #6
            Thanks so much for your assistance Joseph. The more I get into Stata, the more I am appreciating it's capabilities.

            Comment

            Working...
            X