Variable defined in "forval" loop does not exist

[Ekaterina Shpanenberg]

Dear Statalist members,

Could you please help me figure out what's wrong with my loop below?
I define a variable "i" for a loop, and Stata complains that this variable does not exist..

tempname myresults
postfile `myresults' fund intercept using myresults.dta
local N = _N
quietly foreach v of varlist T {
forval i=37/N {
gen temp = `v'[_n] if _n<i & _n>=i-36
gen temp2 = mkt[_n] if _n<i & _n>=i-36
xi: regress temp temp2
post `myresults' (`v') (`=_b[_cons]')
drop temp
drop temp2
}
}
postclose `myresults'

I receive an error: i not found
r(111);

How can it be if I define this variable at the beginning of the loop?
Thanks a lot!

[Stephen Jenkins]

In the line that you emboldened ("forval i=37/N"), I presume you wish to refer the local macro you defined 2 lines previously. In which case, refer to " `N' ". Two further recommendations. (1) Please read the Forum FAQ and note the strong recommendation to post code within CODE delimiters (you'll understand what I mean if you read the FAQ). It helps legibility a lot. (2) I would get into the habit of indenting lines of code within { ... }. Logical structure is revealed visibly; and again, the code is generally more legible.

[Nick Cox]

There are several errors in your code. In fact I am surprised that Stata didn't throw you out earlier.

tempname myresults 
postfile `myresults' fund intercept using myresults.dta
local N = _N 
quietly foreach v of varlist T {

Looks legal, but you don't need to define a loop over a single variable.

forval i=37/N {

Stata won't evaluate N on the fly. You need quotation marks.

forval i=37/`N' {

gen temp = `v'[_n] if _n<i & _n>=i-36 
gen temp2 = mkt[_n] if _n<i & _n>=i-36 
xi: regress temp temp2

This could be much simpler

local j = `i' - 36 
local k = `i' - 1 
regress T mkt in `j'/`k'

You don't need the variables temp temp2 at all.
in qualifiers are always faster than if qualifiers.
xi is superfluous here.

 
post `myresults' (`v') (`=_b[_cons]')

That looks wrong. The local macro v here is a variable name. You can't post an entire variable to a single observation in a new file.

drop temp
drop temp2

Unnecessary as already explained.

At this point my guess is that you are reinventing what is already available as rolling, so do look at its help.

[Ekaterina Shpanenberg]

Thanks a lot for your help!
As you can probably see, I'm new to Stata (MatLab, to which I'm used to, has different intuition).
Nick, you're right about rolling...Thanks, I did not know this function exists.

post `myresults' (`v') (`=_b[_cons]')

locates the first observation into the first column of "myresults"..I wanted to locate there the name of the variables, but could not do this..Could you maybe give hints how can I do that?
Thanks a lot!

[Nick Cox]

rolling is strictly a command, not a function.

Your code is a loop over one variable, so the variable name is known already. But double quotes would flag that you want to post a string, in this case a variable name. Examples at (say) http://www.stata-journal.com/sjpdf.h...iclenum=pr0036

[Ekaterina Shpanenberg]

Nick, thanks again!
And if I add some variables so that my loop runs over several variables, how can I add columns to the right so that the regression results are saved not only over the rolling window, but also over different variables in the same "myresults" file?

[Nick Cox]

You are asking how to generalise or extend code that you don't show us.

Also, it's not clear how far you are exploiting rolling as recommended and not reinventing established code.

I don't want to speculate on what your code might be. Please back up and study http://www.statalist.org/forums/help on how to ask questions: #12 particularly applies. (Looking around threads that go well could also be instructive.)

You've got a history of programming in MATLAB; similar advice would (I imagine) apply in MATLAB forums.

[William Lisowski]

Welcome to Statalist, Ekaterina.

You will find that Stata does indeed have much different behavior than Matlab, or SAS, or SPSS, or R. Really, each of these has much different behavior than the others.

To build up your Stata intuition rapidly, you can do as I did when I began using Stata in a serious way, coming to it from SAS. I started as have others here, by reading my way through the Getting Started with Stata manual relevant to my setup. Chapter 18 then gives suggested further reading, much of which is in the Stata User's Guide, and I worked my way through much of that reading as well. All of these manuals are included as PDFs in the Stata installation (since version 11) and are accessible from within Stata - for example, through Stata's Help menu. The objective in doing this was not so much to master Stata as to be sure I'd become familiar with a wide variety of important basic techniques, so that when the time came that I needed them, I might recall their existence, if not the full syntax, and know how to find out more about them in the help files and manual. Or using your useful way of framing the issue, I built up my intuition about Stata.

Stata supplies exceptionally good documentation that amply repays the time spent studying it. The path I followed surfaces the things you need to know to get started in a hurry and to work effectively.

Good luck!

[Ekaterina Shpanenberg]

Dear all,
I would try to rephrase my question(s), taking your suggestions into account.

1. A question about "postfile":

I have the following code:

tempname myresults 
postfile `myresults' fund intercept1 using myresults.dta, replace
local N = _N 
quietly foreach v of varlist O T BG {
    forval i=37/`N' {
    local j = `i' - 36 
    local k = `i' - 1 
    regress `v' mkt in `j'/`k'
    post `myresults' (A[`i']) (`=_b[_cons]')
}
}
postclose `myresults'

Using this code, Stata writes the values of intercepts only for the last regression in a new file "myresults". I would rather want that it saves regression intercepts for each regression, adding columns to the right in "myresults" file. As far as I discovered, I have to save the intercepts of each regression using "local" and then write them all into "myresults". Thus, I tried to modify the code this way:

tempname myresults 
postfile `myresults' fund intercept1 using myresults.dta, replace
local N = _N 
quietly foreach v of varlist T BG {
    forval i=37/`N' {
    local j = `i' - 36 
    local k = `i' - 1 
    regress `v' mkt in `j'/`k'
    local topost[`i'] = (`=_b[_cons]')
}
local topost2 (`v') = (topost)
post `myresults' (`topost') (`=_b[_cons]')
}
postclose `myresults'

This code does not do what I planned and gives me an error.
Could you please guide me how can I add columns into "myresults" table so that Stata saves not only intercepts of the one regression, but all of them (indicated in "varlist").

2. The question about changing my code to "rolling"

When I try to change the regression part of my code to "rolling", I have a problem with -tsset-. I do the following code:

tempname myresults 
postfile `myresults' fund intercept1 using myresults.dta, replace
local N = _N 
quietly foreach v of varlist T {
    forval i=37/`N' {
    local j = `i' - 36 
    local k = `i' - 1 
    tsset `v'
    rolling _b, window(36): regress `v' mkt
    post `myresults' (`topost') (`=_b[_cons]')
}
}
postclose `myresults'

and receive an error "time variable not set, use -tsset varname ...-", because I have decimal numbers in T variable (returns). How can I overcome this issue?

Thanks a lot in advance for your patience and your help!

[Nick Cox]

I'll peel off the last question.

tsset requires minimally a time variable, usually some kind of date variable, and if applicable also a panel identifier.
It seems that you don't have panel data.
You should not feed it a response variables such as returns. See also help tsset

Also, a major point of rolling is that it automates the loop and makes the post machinery redundant.

Here is a minimal example you can run:

* sandbox to play
clear
set obs 360
gen t = ym(1985, 12) + _n
format t %tm
tsset t
gen x = _n
gen y = 2 + 3 * x
save mydata

rolling, window(36) saving(rolling_results): regress y x

[Ekaterina Shpanenberg]

Dear Nick,

thanks, I succeeded in "rolling", my code shrank to just 4 lines compared to 11 lines before using "rolling".

gen t = ym(1979, 12) + _n
foreach v of varlist T {
    tsset t
    rolling, window(36) saving(rolling_results): regress T mkt
}

Then, if I would like to add more than 1 variable in "varlist", I receive an error "file rolling_results.dta already exists". Obviously Stata saves the results only for one regression. In order to do rolling in a loop over several variables, should I then create many "rolling results" for each regression first and then merge them together to have a one matrix with the results for all variables? Or there is a smarter way to do that?

Thanks!

[Nick Cox]

I'd run rolling repeatedly and merge the resulting files. You must specify a new results file each time around.

Note that you don't need to tsset each time around the loop.

[Nick Cox]

Here is a minimal example:

* sandbox to play
clear
set obs 360
gen t = ym(1985, 12) + _n
format t %tm
tsset t
gen x = _n
gen y1 = 2 + 3 * x
gen y2 = 4 + 5 * x 

foreach v of var y1 y2 { 
    rolling `v'_i=_b[_cons] `v'_g=_b[x], window(36) saving(rolling_`v', replace): regress `v' x
} 

use rolling_y1, clear 
merge 1:1 start using rolling_y2

[Ekaterina Shpanenberg]

Dear Nick,

the approach you mentioned for merging does not work, because in "rolling" the variables in the files "rolling_`v'" have the same names.
So, merge does not add the regression results from other "rolling_" files into the right columns. To avoid that, I think I have to indicate Stata to name the variables in rolling regression to add smth like _1, _2 and so on.

Could you please specify how can I do in the regression?

Thanks a lot.

[Nick Cox]

The code in #13 works. I tried it before posting.

You tried some variation on it which you do not show us. So, you broke the code but I cannot tell you how you broke it. I can't see your code. No one else can see your code.

If you want more advice, you must show us the code you tried. Otherwise, my answer is the same. The approach in #13 is sound.