Next iteration in a loop triggered by "exit" within a program

[Emmanuel Milet]

Dear Statalist users,

This post is related to earlier posts on how the "exit" command works in foreach of forvalues loops. I have read these posts, and am still stuck with what I believe is a a somewhat different problem. My apologies if the answer already exists somewhere on this forum, I haven't been able to find it.



Here is what I am trying to do:

I am looping over many datasets. Each time a dataset is loaded, I check for the existence of a predefined set of variables (always the same). If one of these variable is missing, I exit the datafile, and load the next one. If not, the program continues.
I want to test the existence of a large number variables. Here is the code that I have, which works perfectly fine (I only show the code where I test the existence of 3 variables).


fs *.dta
foreach file in `r(files)' {
use `file', clear
*
capture confirm variable ccode
if _rc!=0 {
exit
}
capture confirm variable year
if _rc!=0 {
exit
}
capture confirm variable lstatus
if _rc!=0 {
exit
}
dis "the dataset is fine"
}
*
*
This code works just fine: If the first variable "ccode" does not exist in the first dataset then Stata exits this loop and loads the second dataset, etc.
To space space, I wrote a small program to check the existence of a variable:

program define ExistenZ
args var_tocheck
capture confirm variable var_tocheck
if _rc!=0 {
dis in red "the variable `var_tocheck' does not exist: exit the loop"
exit
}
end
*
*
I then call the program in the loop:

fs *.dta
foreach file in `r(files)' {
use `file', clear
*
ExistenZ ccode
ExistenZ year
ExistenZ lstatus
dis "the dataset is fine"
}

The problem is that if any of the variable "ccode" for instance does not exist in the dataset, the code keeps running and tests for the existence of the variable "year", instead of exiting the loop and loading the next dataset. I am quite puzzled by this, as I do not understand why the code does what I want in the first example, and not in the second.

If someone can explain to me what is going, that would be great.

Best regards,
Emmanuel Milet

[Carole J. Wilson]

It seems that the program version is doing as expected: exiting the program (not the do-file) when a variable does not exist. One way to avoid this is for your program to accept a list of variables. If it encounters a missing variable, it will exit the program and no longer process the list. Another approach would be to write a loop in your do-file performing the same steps as you started with:

fs *.dta
foreach file in `r(files)' {
  use `file', clear
  *
    foreach var in ccode year lstatus {
        capture confirm variable `var'
        if _rc!=0 {
        exit
         }

  dis "the dataset is fine"
  }

[Nick Cox]

For use of Carole's code, make sure that you download fs from SSC.

[Carole J. Wilson]

Yet another option is to include an error message in your program:

prog drop ExistenZ
program define ExistenZ
args var_tocheck
capture confirm variable var_tocheck
if _rc!=0 {
local rc=_rc
dis in red "the variable `var_tocheck' does not exist: exit the loop"
error `rc'
}
end

[Emmanuel Milet]

Dear Carole, Nick.
Thanks a lot for these very quick answers.
I guess I'll stick to your suggestion Carole. However, I fell that if my program allows for a list of variables, then if it encounters a missing variable, it will exit the program, but still remain in the loop, which I don't want it to.

Best,
Emmanuel

[Emmanuel Milet]

Carole,
Actually, your first suggestion (a loop within the main loop) does not do what I want. Stata exits the loop where it checks for the existence of variables as soon as it finds a missing variable, but what follows keep running, which is problematic in my case (I do operations using the very same variable, so I have to make sure they exist).

As for your suggestion to include an error message, Stata tells me that the syntax is invalid (the last line is apparently where the syntax is wrong).

By the way, I am using Stata 14.

[Sergiy Radyakin]

Why is the inner loop necessary?

local probe "price turn weight"

sysuse auto
ds
local all `"`r(varlist)'"'
local result: list probe in all
if (`result'==1) {
  // ..process file..
}

Best, Sergiy Radyakin

[Carole J. Wilson]

I'm not completely sure what exactly you want to exit. Does this do what you want? Note that there was an error in your first program (and then mine).

capture confirm variable var_tocheck

should have read

capture confirm variable `var_tocheck'

Give this a try. It exits the program (which accepts a list of variables) but not the do file, so you can loop through the files. In this case, the outer loop (display Hello then display World) substitutes for your file loop.

clear
gen year=.

cap prog drop ExistenZ
program define ExistenZ
syntax anything 
local rc 0
foreach var of local anything {
if "`rc'"!="0" exit
cap confirm variable `var'
local rc=_rc
if `rc'!=0 {
    dis in red "the variable `var' does not exist: exit the loop"
    exit
    }
if `rc'==0 di as text "`var' exists"
    }
end


foreach x in Hello World {
    ExistenZ ccode year lstatus    
    
  
di as result "`x'"
}

[Carole J. Wilson]

@Sergiy Radyakin Smart!

[Emmanuel Milet]

Thanks a lot Sergiy, this works perfectly!
I have to look deeper into the code to understand better what it does.

Thanks to all for your quick answers and for fixing my code!

Emmanuel