Getting a "Operation timed out" error when using 'forvalues' command.

[Garreth Gibney]

Hi all,

I am unable to run a 'forvalues' command, and I am unsure as to why I am receiving the error "Operation timed Out".

Please see the code I am running below.

In sum, I am trying to loop over a large number of files in a folder (approx. 2000 files). I have limited the number of values to '5' to be able to test my code. However, once the code works, I will change the '5' to '2000' in the forvalues loop.


**** Stat Code ****

* save a copy so that we can load each observation in the loop below
tempfile files
save "files", replace

* loop over each file and input each csv file into temporary datasets.
forvalues i = 1/5 {

use "files" in `i', clear
global f = dirname + "/" + filename
global d = filename

clear
import delimited using "$f", varnames(1) numericcols(_all)

gen temp_cel = ((temp-32)*5)/9
gen dewp_cel = ((dewp-32)*5)/9
gen RH = (exp(17.625*(dewp_cel/(dewp_cel/(243.04+dewp_cel)))))/(exp(17.625*(temp_cel/(temp_cel/(243.04+temp_cel)))))

keep(date latitude longitude wdsp temp prcp RH)

save "$path/Data/dta_Temperature_df1/$d.dta", replace
}

**** end of state code ****

Thanks!

[William Lisowski]

Added in edit: This is a duplicated post, and there is a separate discussion at the duplicated copy.

https://www.statalist.org/forums/for...values-command

First, does the code work for even a single dataset? If not, then try the following.

* save a copy so that we can load each observation in the loop below
tempfile files
save "files", replace

* loop over each file and input each csv file into temporary datasets.
* forvalues i = 1/5 {
local i 1

use "files" in `i', clear
global f = dirname + "/" + filename
global d = filename

clear
import delimited using "$f", varnames(1) numericcols(_all)

gen temp_cel = ((temp-32)*5)/9
gen dewp_cel = ((dewp-32)*5)/9
gen RH = (exp(17.625*(dewp_cel/(dewp_cel/(243.04+dewp_cel)))))/(exp(17.625*(temp_cel/(temp_cel/(243.04+temp_cel)))))

keep(date latitude longitude wdsp temp prcp RH)

* }

If that fails you'll know which operation timed out. If it works, then you need to return to the original looping code and output some information to figure out where it is failing.

* save a copy so that we can load each observation in the loop below
tempfile files
save "files", replace

* loop over each file and input each csv file into temporary datasets.
forvalues i = 1/5 {

use "files" in `i', clear
global f = dirname + "/" + filename
global d = filename
display `" i = `i' $f $d "'

clear
import delimited using "$f", varnames(1) numericcols(_all)
display " import succeeded "

gen temp_cel = ((temp-32)*5)/9
gen dewp_cel = ((dewp-32)*5)/9
gen RH = (exp(17.625*(dewp_cel/(dewp_cel/(243.04+dewp_cel)))))/(exp(17.625*(temp_cel/(temp_cel/(243.04+temp_cel)))))

keep(date latitude longitude wdsp temp prcp RH)
display " variable tasks succeeded "

save "$path/Data/dta_Temperature_df1/$d.dta", replace
display " save succeeded "
}

[Garreth Gibney]

Thanks for your reply, and apologies for the duplication. it seemed there was an error when I posted.

1. The code works for a single dataset.

2. It appears that the operation is timing out at the import stage for the second file. Upon investigation, there appears to be an issue with accessing these files through the mac "finder". So I think this is a network rather than a coding error.

Thanks for your help!