Capture command didn't work in loops of reghdfe regression

[Stella Chen]

Hi all,

I try to use a loop to regress by stock, but the loop always stops even when I use the capture command:

levelsof gvkey, local(groups)
foreach group of local groups{
    capture reghdfe yvar xvars controlvars if gvkey==`group'
    if c(rc) == 0 {
        outreg2 using "G:\My Drive\research\tables\gvkey", bdec(4) tdec(2) rdec(3) adec(4) alpha(.01, .05, .10) addstat(R-squared, e(r2_a)) ctitle(`group') excel append tstat
        }
    else {
        display as error "Unexpected error in regression"
        exit `c(rc)'
        }    
}

The loop stops with an error like this:

G:\My Drive\research\tables/gvkey.xml
dir : seeout
G:\My Drive\research\tables/gvkey.xml
dir : seeout
G:\My Drive\research\tables/gvkey.xml
dir : seeout
Unexpected error in regression
r(2001);

What's wrong with my code? I read many posts about skipping errors in the loop but couldn't find a solution.

[Andrew Musau]

Stata is just following the instructions of whoever wrote this code.

else {
display as error "Unexpected error in regression"
exit `c(rc)'
}

If you want to discard the above piece of code:

levelsof gvkey, local(groups)
foreach group of local groups{
    capture{
        reghdfe yvar xvars controlvars if gvkey==`group'
        outreg2 using "G:\My Drive\research\tables\gvkey", bdec(4) tdec(2) rdec(3) adec(4) alpha(.01, .05, .10) addstat(R-squared, e(r2_a)) ctitle(`group') excel append tstat
    }
}

[Nick Cox]

I don't do this kind of thing, but if I did I would want to accumulate a record of groups for which the regression didn't work. Here is an untested sketch of one way to do that. Ellipses ... imply whatever code belongs here.

levelsof gvkey, local(groups)

gen bad = . 
local i = 0 

foreach group of local groups { 
      capture { 
             reghdfe ... 
             outreg2 ...
      } 
      if _rc { 
            local ++i 
           quietly replace bad = `group' in `i/ 
      }
} 

list bad if bad < .

[Nick Cox]

 
 quietly replace bad = `group' in `i'

[Clyde Schechter]

I do do this kind of thing. But it has to be done correctly. The point of this kind of code is that a command is being applied to subsets of the data, gvkey levels in this case, and it is anticipated that some of those subsets will have too few complete cases to support the regression command. What is wanted is to output the results when the regression succeeds, simply move on to the next subset if the regression fails due to insufficient (or no) observations, but break with an error message if anything else goes wrong during the regression. So that requires a 3-armed -if-else if-else construction that deals with the case of insufficient or no observations

Now, when I do this sort of thing, I don't export the results to a spreadsheet. Instead I typically save the results either in the original data set, or in another Stata dataset (tempfile or frame). Because of that, I don't need to create a bad list, as suggested by Nick in #3, since I can identify the bad subsets by the absence of output associated with them. But creating a bad list is probably a good idea in O.P.'s situation because it will be harder to identify the bad subsets in a spreadsheet of results. So, you can do both, as follows:

levelsof gvkey, local(groups)
foreach group of local groups{
    capture reghdfe yvar xvars controlvars if gvkey==`group'
    if c(rc) == 0 { // SUCCESSFUL REGRESSION: OUTPUT RESULTS
        outreg2 using "G:\My Drive\research\tables\gvkey", bdec(4) tdec(2) rdec(3) adec(4) alpha(.01, .05, .10) addstat(R-squared, e(r2_a)) ctitle(`group') excel append tstat
        }
    else if inlist(c(rc), 2000, 2001) { // NO, OR INSUFFICIENT OBSERVATIONS: SKIP
        display "gvkey `group' skipped due to insufficient data"
    }
    else {    // UNANTICIPATED ERROR: BREAK
        display as error "Unexpected error in regression: gvkey `group'"      
        exit `c(rc)'
        }    
}

By the way, I do think it is important to have the code abort execution when an unexpected error arises. A drawback of the code in #3 is that it doesn't do that. In fact it doesn't distinguish between the "normal" anticipated problems arising from subsets with insufficient data and other errors. All just get listed as "bad" and execution keeps going even after an unexpected error. Sure, the unexpected error is listed in the "bad list" but if you have a lot of gvkeys, you might not notice a serious error in the midst of it. By forcing a halt to execution, the code her assures that you will not blithely continue a problematic analysis.