forval has trouble with locals?

[Frank Taumann]

I'm trying to write code with some loops where do some data manipulations based on a window. I have trouble getting the code to a point where I just have to change a local with the window length. This works:

sysuse auto, clear
xtset price turn /*just to make sure the format fits, these arent panel variables*/
egen placeholder=mean(turn)
gen basis=round(placeholder) /*a hint how to do this in one line would also be nice ;-) */
drop placeholder
gen keeper=.
forval t = -5/6{
replace keeper=1 if turn==basis+`t'
}

but the following doesn't (you can continue directly with the code). I tried display `min' and the locals are built correctly. Why does Stata not understand these as numbers? I'm a bit dumbfounded.

replace keeper=.
local window 5
local min `window'*-1
local max `window'+1
forval t = `min'/`max'{
replace keeper=1 if turn==basis+`t'
}

In my intuition, all I did was replacing a -5 with a local that is -5. What am I missing?

[Chris Larkin]

I'm pretty sure your problem lies with the backslash tilde combination at the beginning of your local `max'. Stata views /` as an escape sequence. Try instead to turn both your locals into globals, thusly

 
replace keeper=.
local window 5
global min `window'*-1
global max `window'+1
forval t = $min/$max{
replace keeper=1 if turn==basis+`t'
}

Does that help?

[Frank Taumann]

Thanks for answering! No, still the great Stata debug help "invalid syntax" ;-)

[Chris Larkin]

If you can share a snippet of your data using dataex (ssc install dataex) then I can help. I can't see what else is wrong here. It could be something in the variable formatting

[Frank Taumann]

I thought the sysuse auto thing works everywhere? Is the example not reproducible?

[Chris Larkin]

You're completely right Frank! I overlooked your first snippet and the fact it was bringing in the auto dataset.

This code works

replace keeper=.
local window 5
local min `=`window'*-1'
local max `=`window'+1'
forvalues t = `min'/`max'{
    replace keeper=1 if turn==basis+`t'
}

When constructing locals that contain other locals you need to use the `=`localname'' formation to tell Stata that it needs to evaluate the text first to find out what's inside it. This is the case for your locals min and max. As it stood, you were creating a forvalues loop that looked like this when Stata was trying to evaluate it:

forvalues t = 5*-1/5+1{
replace keeper=1 if turn==basis+`t'
}

But the forvalues help file outlines that the ranges should be specified in one of these formats:

                 #1(#d)#2      meaning #1 to #2 in steps of #d    
                 #1/#2         meaning #1 to #2 in steps of 1      
                 #1 #t to #2   meaning #1 to #2 in steps of #t - #1
                 #1 #t :  #2   meaning #1 to #2 in steps of #t - #1

Hence the problem with your code

Alternatively, you could evaluate the min and max locals within the for loop itself, thusly

forvalues t = `=`min''/`=`max''{
    replace keeper=1 if turn==basis+`t'
}

[Nick Cox]

Chris has the main idea here, namely that Frank needs to evaluate the macros before forval can see them as legal input.

(Comments about backslashes make no sense to me here; there are no backslashes and the forward slashes I see are used correctly.)

Let's back up, just to spell out some fundamental points here:

1. All macros contain text strings.

2. Sometimes those text strings have numeric interpretations once they are evaluated.

3. Macros (local and global) are evaluated first, before Stata tries to execute each statement.

The statements

local window 5  
local min `window'*-1

have the net result that the local min contains the text

5*-1

which is not evaluated when it is defined because there is no equals sign. Without an equals sign, the local definition just copies (but once more, copies after macro evaluation has taken place).

If you wanted an evaluation

local min = `window'*-1

would have had the effect you expect of putting the text -5 in the local.

The immediate problem is that Stata sees as the start of your loop (once the macros have been evaluated)

forval t = 5*-1/5+1 {

which isn't one of the legal forms Chris lists. In that context the expressions are not evaluated first unless you use the syntax Chris shows.

Note that display has a dual role of (a) evaluating expressions to the extent possible and then (b) showing the results. So, it is not always a reliable witness to what macros contain. For that, you need macro list, mainly.

Incidentally,

  
egen placeholder=mean(turn)
gen basis=round(placeholder)  
drop placeholder

is a very indirect method to get a rounded mean. Putting constants in variables only to throw away those variables is roundabout. Something like

su turn, meanonly  
local wanted = round(r(mean))

cuts out the need for two entire variables to hold the same constants again and again. Indeed with the extra syntax the macro is dispensable too.

[William Lisowski]

This code is more direct than Chris's code.

local min = `window'*-1
local max = `window'+1

The local (and global) command will evaluate an expression and assign the result to the macro if the expression is preceded by an equal sign. Otherwise, it just assigns the unevaluated expression to the macro, to be evaluated later after the macro has been referenced.

When Frank tried display `min' he ran afoul of the display command's nature of evaluating any expressions in encounters: it behaved as if he had typed display 5*-1 which is why he saw the answer he expected.

In the spirit of Chris's final, elegant solution, Frank could write

forvalues t = `=`window'*-1'/`=`window'+1'{

Note: Crossed with Nick's more comprehensive post.

[Nick Cox]

Revisiting the original code:

sysuse auto, clear
xtset price turn 
su turn, meanonly 
gen keeper = inrange(turn, r(mean) - 5, r(mean) + 6)

isn't exactly the same but may suggest some ways forward.

[Chris Larkin]

Thanks for clarifying my very haphazard attempt to explain the equals operator.

William and Nick's treatments are much more precise and comprehensive than my own

[Frank Taumann]

Thank you for the comprehensive answers you three, this really improved my understanding both of macros and of forvalues. That will help me a lot in the future working with locals. Funny this Stata world. You can do the most complicated panel regressions in a split second but when you want to do something with a single scalar, you start thinking about workarounds and different evaluation methods of different commands

Thanks again!

[Nick Cox]

Fine, but I underline that it's not clear that you need locals or loops at all. Your code seems to be about identifying a window about the mean date for panel data, which is obtainable directly.

[Frank Taumann]

wow. correct. inrange does the trick. I guess I'm thinking to much in the R logic still and make things unnecessarily complicated. Thanks for pointing this out.

edit:

final solution based on your posts:

sysuse auto, clear
xtset price turn
su turn, meanonly
local window 5
gen keeper = inrange(turn, r(mean) +`=`window'*-1', r(mean) + `=`window'+1')

[Nick Cox]

Negation is simpler than multiplying by -1. The trick to macros is to think how Stata will see the result of macro evaluation. Consider

 
 gen keeper = inrange(turn, r(mean) -`window', r(mean) + `=`window'+1')

Incidentally, why an asymmetric window?

[Frank Taumann]

Thanks.

A simple diff in diff exercise where I expect my treatment to have a lag but not always so I'm getting rid of both the t and t+1.