Reducing number of steps in a forval loop

[Attaullah Shah]

I have the following loop, which intends to calculated WLAT, that is average returns of to top 3 firms minus average returns of bottom three firms. I have to use the aWT`i' and aLT`i' becuase I want to spread the results across all observations. Can this code of reduce or improved to increase efficiency in terms of speed or avoiding the generation of aWT`i' and aLT`i' . Sample data is attached.

            forval i=1(1)600{
            local j=`i'+1
            qui egen LT`i'=mean( R`j') if  pf`i'<4
            qui egen WT`i'=mean( R`j') if  pf`i'>7& pf`i'!=.
            qui egen aWT`i'=mean( WT`i')
            qui egen aLT`i'=mean( LT`i') 
            qui gen WLAT`i'= aWT`i'- aLT`i' 
            }
           keep WLAT*

[Clyde Schechter]

I'm not sure if efficiency is really the issue: even 2400 -egen-s are unlikely to take a noticeable length of time unless your data set is gargantuan. But, the code can be improved (and, incidentally, made faster) by noticing that the variables you are creating are all, in fact, constants! So those are better stored as scalars or local macros. In addition, -egen, mean()- has a fair amount of computational overhead that is worth the trouble when you are using it with by-groups, but not when you are just taking a mean of something over the whole data set. So, instead of egen LT`i' = mean(R`j') if pf`i' < 4 you can more efficiently do

summ R`j' if pf`i' < 4, meanonly
local LT`i' = r(mean)

But I think I'm missing something in your code. The generation of aWT`i' and aLT`i' make no sense to me, because LT`i' and WT`i' are themselves constants. You allude to wanting to spread these to all observations. But as noted, it doesn't seem there is any reason to record these results in any observation in the data set at all, as they are constants.

You can similarly calculate WT`i'. And then you can just subtract them to get a local macro WLAT`i'.

At that point, if there is really reason to make variables out of these constants you can always do

forvalues i = 1/600 {
gen WLAT`i' = `WLAT`i''
}

and you will have 600 (constant) variables to work with.

By the way, I noticed that your data set does not contain variables like R2,..., so none of this is going to work on that data in any case. Perhaps you meant to work with variables Ri_RmP* instead of R*?

[Attaullah Shah]

Excellent suggestions Clyde Schechter. You are right about the variable names, in the code I reduced the name from Ri_RmP to Ri for improving readability, but forgot to change it in the data set. I have few questions regarding your suggestions.

1. How many local macros can be defined , I mean what is the limit?

2. I need the results and names of these 600 defined macros in two columns, how can that be done.

You are right about the constant nature of the variables. Previously I would convert them to variables, then would drop duplicates, then would xpose to convert them to columns, but now your suggestions have eliminated the need for doing this. Thanks for your time and consideration.

[Clyde Schechter]

I'm not aware of any limit on the number of macros: I suspect that any limits reflect the amount of available memory rather than a Stata constraint. In any case, I have run code myself that has generated tens of thousands of local macros and encountered no problems.

I'm not completely sure I understand the two-column display you are looking for, but perhaps it is this:

drop _all
set obs 600
gen name = ""
gen value = .
forvalues k = 1/600 {
    replace name = "WLAT`k'" in `k'
    replace value = `WLAT`k'' in `k'
}

[Attaullah Shah]

You have cracked the problem for me. It is so great of you. Thanks once again.