Multiple loops: Predict residual of regression

[Pascal Meier]

Dear Statalist community

I have multiple loops in my code to calculate the residuals from a regression for a certain group. To give you context: I want to calculate discretionary firm accruals (the residuals) by estimating nondiscretionary accruals (predicted values). It is on the firm-year-quarter level.

For each subgroup (denoted by sic_2_n) - subgroup (denoted by year_quarter) combination, I estimate this accrual regression (see code below). However, I constantly get the error message "r(2000) no observations".

There are lots of missing values in my variables. I do not want to clean the data first (i.e., delete all observations with missing values in any of the variables) because the regression should be based on the whole universe. My understanding is that missing observations should not matter because they do not enter the regression (I get the same error message when I delete all the observations with missing values).

If there are too few observation for this subgroup-subgroup-combination (sic_2_n & year_quarter), it should just proceed with the next one. I tried to build this additional loop with

if c(rc)==0

but it seems to fail.

Additionally, I only want to predict the residual if the regression is based on at least 10 observations (hence, this condition after predict with

 if e(N)>=10

Do you have any suggestions what this error message refers you? Probably with my data, but I do not get it - even after applying it on a cleaned subset. A excerpt of my data is found at the bottom.

As always, help is highly appreciated.

Best regards
Pascal

 levelsof sic_2_n, local(sic_reg)
gen absmjda1=.
foreach i of local sic_reg {
    levelsof year_quarter, local(yearquarter_reg)
        foreach p of local yearquarter_reg {
            regress acc_1 inverseassets adj_deltasales ppe i.quint_roa i.quint_mtb i.quint_salesgrowth if sic_2_n==`i' & year_quarter==`p'
            if c(rc)==0 { // everything is okay
                predict resid if e(N)>=10, resid
                replace absmjda1=resid if sic_2_n==`i' & year_quarter==`p'
                drop resid
            }
            else if !inlist(c(rc), 2000, 2001) { // Unexpected error
                display as error "Unexpected error in regression"
                exit c(rc)
            }        
    }
}

* Example generated by -dataex-. For more info, type help dataex
clear
input long gvkey_n float(sic_2_n year_quarter acc_1 acc_2 adj_deltasales inverseassets ppe) byte(quint_roa quint_mtb quint_salesgrowth) float absmjda1
1003 57 108            .            .             .   .06855889         . 5 3 4 .
1003 57 109            .            .             .   .06364157         . 2 3 3 .
1003 57 110            .            .             .   .05931902         . 2 3 2 .
1003 57 111     .0519323    .02212466             .   .05776674 .13269019 2 3 2 .
1003 57 112   .025620246    .01732951             .   .06233637         . 2 2 1 .
1003 57 113    .02885192   .013471503             .    .0545405         . 1 2 2 .
1003 57 114   .033642884   .005950064             .   .05560808         . 1 3 2 .

88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110
>  111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129
> 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 1
> 49 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 16
> 8 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187
>  188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206
> 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 2
> 26 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 24
> 5 246 247 248 249 250 251
no observations
r(2000);

[Nick Cox]

You need

capture regress

as if there are no observations, then the regress will fail and you will never get to the next command.

I would also move the check for 10 or more observations used

 if c(rc)==0 & e(N) >= 10 

rather than putting e(N) >= 10 as a condition on predict where it will pointlessly be evaluated for each observation.

Yet more: do you really want to be thrown out of the loop as soon as one problematic group is encountered?

[Pascal Meier]

You need

capture regress

as if there are no observations, then the regress will fail and you will never get to the next command.

I would also move the check for 10 or more observations used

if c(rc)==0 & e(N) >= 10 

rather than putting e(N) >= 10 as a condition on predict where it will pointlessly be evaluated for each observation.

Yet more: do you really want to be thrown out of the loop as soon as one problematic group is encountered?

Dear Nick,

thanks for your fast reply and the great suggestions - the code seems to be working now!

What I want to do with the else statement: It should just proceed with the next group, i.e., the next sic_2_n - year_quarter combination, if there is a problem with the focal sic_2_n - year_quarter combination - isn't that what the code is doing?

Best regards,
Pascal

[Nick Cox]

This is a sketch, quite untested.

levelsof sic_2_n, local(sic_reg)
levelsof year_quarter, local(yearquarter_reg)
gen absmjda1=.
local vars acc_1 inverseassets adj_deltasales ppe i.quint_roa i.quint_mtb i.quint_salesgrowth

foreach i of local sic_reg {
    foreach p of local yearquarter_reg {
        capture regress `vars' if sic_2_n==`i' & year_quarter==`p'
        if c(rc)==0 { // everything is okay
            if e(N) >= 10 {
                predict resid, resid
                replace absmjda1=resid if sic_2_n==`i' & year_quarter==`p'
                drop resid
            }
            else di "`i'; `p': number of observations = " e(N)
       }
       else { // Unexpected error
            di "`i'; `p': " error _rc
       }
   }        
}

[Clyde Schechter]

It's very rare that I take issue with Nick, but here we are.

Yet more: do you really want to be thrown out of the loop as soon as one problematic group is encountered?

It depends on what is meant by "problematic." The relevant part of the code written by O.P. is

else if !inlist(c(rc), 2000, 2001) { // Unexpected error
     display as error "Unexpected error in regression"
    exit c(rc)
}

The is design is the one that I use in circumstances like this. In fact, the code looks very much like it was adapted from code I have posted in other threads seeking to do multiple regressions and not get tripped up by the subgroups with insufficient or no observations.

There are two kinds of errors that can occur with -regress-. The errors that return c(rc) = 2000, or 2001 are no, or insufficient observations, respectively. These are errors that we expect to arise due to acceptable gaps in the data. When those occur, this -else if- condition is false and the -exit- is not executed. The code proceeds to the next iteration of the loop. On the other hand, any error other than those two will lead to being thrown out of the loop. I think that is appropriate program design. I believe that any unanticipated error condition should cause a program to terminate.* It signals something wrong with the data or the code. And if something is discovered to be wrong in one particular subset of the data, there is a reasonable probability that it also affects some of the other data, parts of the data that are wrong but don't happen to be wrong in ways that trigger an exception. Similarly, if the problem is that the code is incorrect, it may well be that the results generated so far, and to be generated in subsequent iterations, will be incorrect, just in ways that don't happen to cause Stata to throw an error code.

So it makes sense to stop and investigate the problem(s) and fix it (them) rather than proceeding.

Added: *Obviously, this principle would not apply to unanticipated errors in real-time programs that are controlling sensitive or critical processes. Such programs need to include "fall back" code that will keep the process limping along as well as possible, or, find a way to exit gracefully. But within the context of research data analysis, I do believe that any detection of an error condition that is not anticipated and understood to represent normal conditions implies that the results generated so far, and that would be generated by continuing, cannot be trusted. Program termination is, in my view, the appropriate step in this situation.

[Pascal Meier]

It's very rare that I take issue with Nick, but here we are.



It depends on what is meant by "problematic." The relevant part of the code written by O.P. is

else if !inlist(c(rc), 2000, 2001) { // Unexpected error
display as error "Unexpected error in regression"
exit c(rc)
}

The is design is the one that I use in circumstances like this. In fact, the code looks very much like it was adapted from code I have posted in other threads seeking to do multiple regressions and not get tripped up by the subgroups with insufficient or no observations.

There are two kinds of errors that can occur with -regress-. The errors that return c(rc) = 2000, or 2001 are no, or insufficient observations, respectively. These are errors that we expect to arise due to acceptable gaps in the data. When those occur, this -else if- condition is false and the -exit- is not executed. The code proceeds to the next iteration of the loop. On the other hand, any error other than those two will lead to being thrown out of the loop. I think that is appropriate program design. I believe that any unanticipated error condition should cause a program to terminate.* It signals something wrong with the data or the code. And if something is discovered to be wrong in one particular subset of the data, there is a reasonable probability that it also affects some of the other data, parts of the data that are wrong but don't happen to be wrong in ways that trigger an exception. Similarly, if the problem is that the code is incorrect, it may well be that the results generated so far, and to be generated in subsequent iterations, will be incorrect, just in ways that don't happen to cause Stata to throw an error code.

So it makes sense to stop and investigate the problem(s) and fix it (them) rather than proceeding.

Added: *Obviously, this principle would not apply to unanticipated errors in real-time programs that are controlling sensitive or critical processes. Such programs need to include "fall back" code that will keep the process limping along as well as possible, or, find a way to exit gracefully. But within the context of research data analysis, I do believe that any detection of an error condition that is not anticipated and understood to represent normal conditions implies that the results generated so far, and that would be generated by continuing, cannot be trusted. Program termination is, in my view, the appropriate step in this situation.

Dear Nick and Clyde,

thanks for your comprehensive answers, as always - this really helped me. Yes, I adapted the code from code used in an older project of mine some years ago, which was probably inspired by the forum here. Clyde's reasoning re exiting sounds plausible for my needs in this project.

Thanks again for your help.

Best regards,
Pascal