Loop error when saving regression coefficients

[Nils Holstein]

Dear Statalisters,

I am having trouble in storing regression constants of my OLS Regressions. I want to compute daily regressions for each company in my sample; obs_id identifies company and date.

I tried to build the following loop, somehow the saved coefficient is the same for the whole sample.

foreach x in obs_id {
    newey retRF MktRF SMB HML if obs_id == `x', lag(0)
    mat A = e(b)
    quietly replace constant = A[1, 4] if obs_id == `x'
    matrix A = .
    }

I would appreciate any comments or ideas on my code. I also tried statsby, but my sample is too large to use that command.

Thanks in advance!


Kind regards,

Nils

[Roman Mostazir]

If obs_id is numeric, you need to use 'forvalues' instead `foreach'

[Nils Holstein]

Thank you for the reply, unfortunately it does not solve the problem. I don't get any error message, I just get one coefficient value for the whole sample, so I suspect that stata does not differenciate between time and company.

I use a similar loop in saving regression residuals and tried to reproduce it on the coefficients.

foreach c in obs_id {
    newey retRF MktRF SMB HML if obs_id == `c', lag(0)
    predict temp, res
    replace nwresid = temp if obs_id == `c'
    drop temp
    }

[Nick Cox]

Cross-posted in slightly different form and answered at https://stackoverflow.com/questions/...l-in-variables

Our policy on cross-posting is explicit. You are asked to tell us about it. Do please read and act on the FAQ Advice which all posters are prompted to read before every post.

I didn't see this thread before now, but the fallacy here is explained in my Stack Overflow post. foreach doesn't look inside variables to detect their distinct values. I don't know what led you to believe otherwise, but perhaps it was some similar but not identical construct in other software.

How many observations for each distinct obs_id?

[Nils Holstein]

Cross-posted in slightly different form and answered at https://stackoverflow.com/questions/...l-in-variables

Please accept my apology, I will act according to this policy in the future.

For completeness, this is my final solution to the extract the regression constant.

      
sum obs_id, meanonly

forvalues x = 1/`r(max)' {
        newey retRF MktRF SMB HML if obs_id == `x', lag(0)
        mat A = e(b)
        quietly replace const = A[1, 4] if obs_id == `x'
        capture matrix drop A
        }

Thank you all for your help.

[Nick Cox]

Some variant on

 
 statsby, by(obs_id) : newey retRF MktRF SMB HML, lag(0)

would be more direct.

[Nick Cox]

Note that the suggestion of statsby and also runby (SSC) was made previously in https://stackoverflow.com/questions/...l-in-variables

[Robert Picard]

Here's a fully worked out example using runby (from SSC) with a synthetic demonstration dataset. Note that runby does not carry over the xtset declaration to the by-group data subset so the monthly_newey program includes a tsset to declare the by-group's data a time series.

* create a demonstration dataset; 20 companies with 365 days
clear all
set seed 123
set obs 20
gen long permno = _n
expand 365
bysort permno: gen date = mdy(12,31,2015) + _n
format %td date
gen retRF = runiform()
gen MktRF = runiform()
gen SMB = runiform()
gen HML = runiform()
xtset permno date

* generate a monthly date
gen mdate = mofd(date)
format %tm mdate

program monthly_newey
    tsset date
    newey retRF MktRF SMB HML, lag(0)
    gen b_cons = _b[_cons]
end
runby monthly_newey, by(permno mdate) status