Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Using required error

    Dear All,
    I want to download a .csv file from a website and process it. The files are available as separately for each month and each variable. So I am trying to write a short loop but get an error:

    Code:
    . clear
    
    . /*import delimited "$download_site_file_path/2020/05/prcp-202005-ste-prelim.csv"
    >         drop v1 v2
    >         rename (v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19 v20 v21 v22 v23 v24 v25
    >  v26 v27 v28 v29 v30 v31 v32 v33 v34 v35 v36 v37) ///
    >         (Statename Year Month cli_measure day1 day2 day3 day4 day5 day6 day7 day8 day9 day10 day11 d
    > ay12 day13 day14 day15 day16 day17 day18 day19 ///
    >         day20 day21 day22 day23 day24 day25 day26 day27 day28 day29 day30 day31)
    > drop cli_measure
    > reshape long day, i(Statename) j(Day)
    > rename day prcp
    > recode prcp -999.98999=.
    >         */
    . forval M = 1/1{
      2.     foreach var in prcp {
      3.         import delimited "$download_site_file_path/2020/0"`M'"/"`var'"-20200"`M'"-ste-prelim.csv"
      4.         drop v1 v2
      5.         rename (v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19 v20 v21 v22 v23 v24 
    > v25 v26 v27 v28 v29 v30 v31 v32 v33 v34 v35 v36 v37) ///
    >         (Statename Year Month cli_measure day1 day2 day3 day4 day5 day6 day7 day8 day9 day10 day11 d
    > ay12 day13 day14 day15 day16 day17 day18 day19 ///
    >         day20 day21 day22 day23 day24 day25 day26 day27 day28 day29 day30 day31)
      6.         drop cli_measure
      7. reshape long day, i(Statename) j(Day)
      8. rename day prcp
      9. recode prcp -999.98999=.
     10.         save "$rawdata_file_path\`var'_2020`M'.dta", replace
     11. }
     12. 
    . }
    using required
    r(100);
    
    end of do-file
    
    r(100);
    What am I doing wrong?

    Many thanks in advance for your help.
    Sincerely,
    Sumedha.

  • #2
    It's hard to tell given what you're showing us! I would suggest doing some experiments to check that the macros in the expression

    Code:
     
     "$download_site_file_path/2020/0"`M'"/"`var'"-20200"`M'"-ste-prelim.csv"
    are being expanded as you intend. Perhaps you need also to check out "compound double quotes".

    Comment


    • #3
      Thank you for your help. How may I check that its expanded correctly?

      Comment


      • #4
        I believe your use of quotation marks in file paths is confusing Stata, causing it to issue a confusing error message. Try replacing
        Code:
        import delimited "$download_site_file_path/2020/0"`M'"/"`var'"-20200"`M'"-ste-prelim.csv"
        with
        Code:
        import delimited "$download_site_file_path/2020/0`M'/`var'-20200`M'-ste-prelim.csv"
        (removing all the double-quote marks between the outermost pair) and replacing
        Code:
        save "$rawdata_file_path\`var'_2020`M'.dta", replace
        with
        Code:
        save "$rawdata_file_path/`var'_2020`M'.dta", replace
        (replacing the back-slash with a forward-slash).

        To address your question in post #3,
        Code:
        . global download_site_file_path /a/b
        
        . local var prcp
        
        . local M 1
        
        . local ex1 "$download_site_file_path/2020/0"`M'"/"`var'"-20200"`M'"-ste-prelim.csv"
        
        . local ex2 "$download_site_file_path/2020/0`M'/`var'-20200`M'-ste-prelim.csv"
        
        . macro list _ex1 _ex2
        _ex1:           /a/b/2020/0"1"/"prcp"-20200"1"-ste-prelim.csv
        _ex2:           /a/b/2020/01/prcp-202001-ste-prelim.csv

        Comment


        • #5
          Prof. Lisowski,
          Thank you so much. This is super helpful. Unfortunately it does not solve my problem completely yet. So, using your code i figures that _ex2 in #4 is how i should write it. I have also tried to simplify it by removing one of the loops. But still, I get an error.

          Code:
          .         
          .         forval M=1/5{
            2.             clear
            3.         import delimited "$download_site_file_path/2020/0`M'/prcp-20200`M'-ste-prelim.csv"
            4.         drop v1 v2
            5.         rename (v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19 v20 v21 v22 v23 v24 
          > v25 v26 v27 v28 v29 v30 v31 v32 v33 v34 v35 v36 v37) ///
          >         (Statename Year Month cli_measure day1 day2 day3 day4 day5 day6 day7 day8 day9 day10 day11 d
          > ay12 day13 day14 day15 day16 day17 day18 day19 ///
          >         day20 day21 day22 day23 day24 day25 day26 day27 day28 day29 day30 day31)
            6.         drop cli_measure
            7. reshape long day, i(Statename) j(Day)
            8. rename day prcp
            9. recode prcp -999.98999=.
           10.         save "$rawdata_file_path/prcp_2020`M'.dta", replace
           11. }
          could not open url
          r(603);
          
          end of do-file
          
          r(603);
          However, it works perfectly when I take out the loop:

          Code:
           clear
          
          . import delimited "$download_site_file_path/2020/05/prcp-202005-ste-prelim.csv"
          (37 vars, 48 obs)
          Not sure why the loop messes it up. I will be grateful for any help you may be able to offer.
          Sincerely,
          Sumedha.

          Comment


          • #6
            Your message is different now: it tells you Stata could not open the URL I will note that it failed for M==1 but you show us a successful command with M==5 so your comparison is unconvincing.

            Since you haven't shared the URL with us there's not much more I can tell you; I can't even try to replicate your problem. Please post in a code block the following commands and their output.
            Code:
            local M 1
            local path "$download_site_file_path/2020/0`M'/prcp-20200`M'-ste-prelim.csv"
            macro list download_site_file_path _path
            import delimited "$download_site_file_path/2020/0`M'/prcp-20200`M'-ste-prelim.csv"

            Comment


            • #7
              Prof. Lisowski,
              Thank you so much for your help. You pointed out that in my working example M=5 whereas in my non-working example M=1. I checked and for some reason the file name on the web resource varied slightly between M=5 and M=1, causing the error. I greatly appreciate your help.
              Sincerely,
              Sumedha.

              Comment

              Working...
              X