Creating and storing residuals in a loop

[Farah Ali]

Hi all

I am trying to run the following code. The purpose is to capture industry and year wise residuals from the stated model.
It works fine. However, when I try to replace the i= "twodigcode"(i.e. two digit codes with one digit codes) i= "onedigcode, Stata still gives the same results. I have also changed the i =1/6. But the results donot change. Can someone please guide where I am going wrong or what am I missing. Thanks in anticipation.

forvalues y = 2015/2017 {   // Define a for/next loop spanning years in sample

               forvalues i = 1/45 { // Define a for/next loop spanning the industries in the sample

               capture: reg   INVESTMENT SALESGROWTH    if y=='Years' & i=='twodigcode' , noconstant    // estimate Jones-type regression silently, within ind-year samples

              

               capture: predict resid`++n' if e(sample), residuals   // save residuals in temporary variable named 'residXXX', and increment the local counter

               capture: replace `rname'=resid`n' if e(sample) // update values of permanent variable with residuals estimated in line above

               capture: drop resid`n'  //drop temporary variable

                              } // NEXT INDUSTRY

               } // NEXT YEAR

[Nick Cox]

This is hard to read and -- more importantly -- hard to follow.

Programmers often develop their conventions about indenting, spacing, commenting and so forth, so I am imposing mine here so that I can think about the code more carefully. (I wouldn't use so many comments, but that's personal taste.)

// Define a for/next loop spanning years in sample
forval y = 2015/2017 {  

    // Define a for/next loop spanning the industries in the sample
    forval i = 1/45 {

        // estimate Jones-type regression silently, within ind-year samples
        capture reg INVESTMENT SALESGROWTH  if y=='Years' & i=='twodigcode', noconstant    

        // save residuals in temporary variable named 'residXXX', and increment the local counter
        capture predict resid`++n' if e(sample), residuals  

        // update values of permanent variable with residuals estimated in line above
        capture replace `rname' = resid`n' if e(sample)

        //drop temporary variable
        capture drop resid`n'  

    } // NEXT INDUSTRY
} // NEXT YEAR

As presented, your code is illegal, so it should not have produced any results at all (except that, as @Jorit points out, capture ate your errors). The following are puzzles.

1. Your loops are defined in terms of local macros y and i but you never refer to them inside either loop. That would imply getting the same results again and again if you get any results at all. Referring to bare y and i isn't right, as those will be interpreted as references to variables (or scalars) with names that are y or i (or abbreviate to such).

2. You are using incorrect punctuation around

 'Years' 'twodigcode'

If those are local macro references you need different quote characters (and where are the local macros defined?).

 `Years' `twodigcode'

3. You use local macros n and rname which may well be defined earlier in your code. If so, that could be fine, but you should explain what's going on.

4. The variable referred to as temporary isn't temporary in Stata's sense. We're not in a programming class, but if we were you'd lose marks for a comment that isn't strictly correct. That's a little deal.

Many people here could suggest better code but they would be helped by knowing more about your data. In particular, it's not clear what your industry and date variables really are named!

My wild guess is that you need something more like

forval y = 2015/2017 {  
    forval i = 1/45 {
        capture reg INVESTMENT SALESGROWTH if Years == `y' & twodigcode == `i', noconstant    
        if _rc == 0 {
             predict resid if e(sample), residuals  
             replace `rname' =  resid if e(sample)
             drop resid
        }
    }
}

[Jorrit Gosens]

Ah, was working on a similar post as Nick.
below is an example that would do what I think you want, with some play data.

Most importantly, where you say 'it runs fine', it really doesn't. Stata doesn't throw an error, but the only reason it does not it because the 'capture' statements at the beginning of each line instruct Stata to continue even if an error is encountered. If I run your exact code in pos #1 with the example created below, Stata encounters no situation at all where instructions are without errors, it then skips over those errors because of the capture statements, and it ends the loop after having done nothing at all.
Don't use capture unless there are really good reasons for it.

clear
webuse grunfeld
replace year=2015
replace year=2016 if time>8
replace year=2017 if time>13
ren invest INVESTMENT
ren mvalue SALESGROWTH
ren year Years
ren company twodigcode

set trace off

local n = 0
gen residuals =.
forvalues y = 2015/2017 {   // Define a for/next loop spanning years in sample
    forvalues i = 1/2 { // Define a for/next loop spanning the industries in the sample
    reg   INVESTMENT SALESGROWTH    if Years==`y' & twodigcode==`i', noconstant    // estimate Jones-type regression silently, within ind-year samples
    predict resid`++n' if e(sample), residuals   // save residuals in temporary variable named 'residXXX', and increment the local counter
    replace residuals=resid`n' if e(sample) // update values of permanent variable with residuals estimated in line above
    *drop resid`n'  //drop temporary variable
    } // NEXT INDUSTRY
} // NEXT YEAR

[lal mohan kumar]

Dear all, I have a problem while running the below loop. Let me demonsterate with a sample example.

[CODE]input str1 firm float (cashflow assets sales) int year float industry
"a" 100 500 300 1991 1
"a" 125 550 410 1992 1
"a" 129 550 350 1993 1
"a" 118 450 216 1994 1
"a" 96 600 175 1995 1
"b" 350 1500 600 1991 1
"b" 560 1675 850 1992 1
"b" 730 1300 755 1993 1
"b" 900 1800 1065 1994 1
"b" 1050 2000 1800 1995 1
"c" 60 120 155 1991 2
"c" -10 120 180 1992 2
"c" 50 160 168 1993 2
"c" 200 150 260 1994 2
"c" -60 140 200 1995 2
"d" 155 230 200 1991 2
"d" 255 398 400 1992 2
"d" 179 398 268 1993 2
"d" 196 423 318 1994 2
"d" 165 300 215 1995 2
end
egen Industry_code=group( industry )
egen firm_iden=group( firm )
xtset firm_iden year,yearly
gen discretionary = .
forval y = 1991(1) 1995 {
forval i = 1(1) 2 {
display `i'
display `y'
capture reg cashflow assets if `i' == Industry_code & `y' == year, noconstant
if c(rc) == 0 {
predict r if `i' == Industry_code & `y' == year, resid
replace discretionary = r if `i' == Industry_code & `y' == year
drop r
}
}
}
** To cross check my residuals are correct or not, I run the below commands
reg cashflow assets if Industry_code==1 & year==1991, noconstant
predict r if Industry_code==1 & year==1991
[CODE]

However, my residuals are starkingly diffferent, why is it so?