Regress by each observation and collect residuals

[Olena Onishchenko]

Dear Stata users

My data is in panel form.Variables are: date, IssueCode, IssueName,YEAR, id, RTN, KOSPI_RTN. I am running a univariate regression by each observation. For each stock number of observations is different.Observations may vary from 01/01/2004 to 30/06/2015.

My regression is reg RTN KOSPI_RTN. All I need from this regression is residulas. My output file should contain date resid IssueCode IssueName

I have tried the following code:

preserve

keep if inrange (date, 01/01/2004, 30/06/2015)
statsby ssr = e(rss), saving(resid, replace) by(date IssueCode id): ///
     reg RTN KOSPI_RTN
restore

* Example generated by -dataex-. To install: ssc install dataex
clear
input float date str12 IssueCode str26 IssueName int Year float id double(RTN KOSPI_RTN)
20046 "KR7000030007" "Woori Bank"      2014 5                     .                      .
20047 "KR7000030007" "Woori Bank"      2014 5   -.13899887480665607    -.00449947382087004
20048 "KR7000030007" "Woori Bank"      2014 5   -.00881062968215492   .0034668441608053315
20051 "KR7000030007" "Woori Bank"      2014 5   .004415018209116693   .0069483819056925895
20052 "KR7000030007" "Woori Bank"      2014 5  -.017778246021283944   .0008437007131357553
20053 "KR7000030007" "Woori Bank"      2014 5                     0  .00031809747683682506
20054 "KR7000030007" "Woori Bank"      2014 5   -.03653374333245595   .0006308463896434597
20055 "KR7000030007" "Woori Bank"      2014 5  -.004662013105811301  -.0006611370283238476
20058 "KR7000030007" "Woori Bank"      2014 5  -.014117881545784932   -.007886508184849338
20059 "KR7000030007" "Woori Bank"      2014 5   .014117881545785022   .0003103496546194288
20060 "KR7000030007" "Woori Bank"      2014 5    .04119575643826733    .002073308430702593
20061 "KR7000030007" "Woori Bank"      2014 5   -.01354422510775726    .008441812085308026
20062 "KR7000030007" "Woori Bank"      2014 5   -.01373019281190202 5.0336879566277434e-06
20065 "KR7000030007" "Woori Bank"      2014 5  -.009259325412796712  -.0038683011846117003
20066 "KR7000030007" "Woori Bank"      2014 5  -.004662013105811301   -.004050741000531012
20067 "KR7000030007" "Woori Bank"      2014 5  -.023641763057040424   -.012965806647386172
20068 "KR7000030007" "Woori Bank"      2014 5  -.004796172263493055   -.015002288026713006
20069 "KR7000030007" "Woori Bank"      2014 5  -.014528100562909744   .0026678493274447314
20072 "KR7000030007" "Woori Bank"      2014 5   .009708814126960903   -.000702746204810658
20073 "KR7000030007" "Woori Bank"      2014 5                     0   -.008487457114453622
20074 "KR7000030007" "Woori Bank"      2014 5  -.014598799421152636  -.0020871181961052677
20075 "KR7000030007" "Woori Bank"      2014 5                     0   -.001400862865555376
20076 "KR7000030007" "Woori Bank"      2014 5    .02421425812059453     .01697240988115261
20079 "KR7000030007" "Woori Bank"      2014 5   .018957913744614207    .006785288484115729
20080 "KR7000030007" "Woori Bank"      2014 5   .009345862418237599  -.0021122378504445094
20081 "KR7000030007" "Woori Bank"      2014 5   .009259325412796726    .003906707356820688
20083 "KR7000030007" "Woori Bank"      2014 5   .013730192811902037    .000795939211727329
20086 "KR7000030007" "Woori Bank"      2014 5   -.07061756721395342   -.010474757927081935
20087 "KR7000030007" "Woori Bank"      2014 5   -.02469261259037152   -.006384910267759445
20090 "KR7000030007" "Woori Bank"      2015 5  -.021223636451626688    .005648070860216349
20093 "KR7000030007" "Woori Bank"      2015 5  -.016478230732384795   -.005564549168018216
20094 "KR7000030007" "Woori Bank"      2015 5  -.019927245652624832    -.01753507095416618
20095 "KR7000030007" "Woori Bank"      2015 5  -.004246290881451097   .0007328186230219282
20096 "KR7000030007" "Woori Bank"      2015 5    .01373502839015247    .010991326127431977
20097 "KR7000030007" "Woori Bank"      2015 5  .0031430093971992214    .010471846774276577
20100 "KR7000030007" "Woori Bank"      2015 5  -.019007964045176632  -.0019502561016178773
20101 "KR7000030007" "Woori Bank"      2015 5  .0010655302020383163  -.0019853631631770044
20102 "KR7000030007" "Woori Bank"      2015 5  -.018270255651669055  -.0018168533764629808
20103 "KR7000030007" "Woori Bank"      2015 5  -.018610279856667713   .0002507968036552422
20104 "KR7000030007" "Woori Bank"      2015 5   -.02121799309484521   -.013681514316287061
20107 "KR7000030007" "Woori Bank"      2015 5   .010106767669774384    .007644962380959802
20108 "KR7000030007" "Woori Bank"      2015 5    .01662088123604028    .008212706429521589
20109 "KR7000030007" "Woori Bank"      2015 5                     0   .0015210158315574853
20110 "KR7000030007" "Woori Bank"      2015 5  -.008830079448272037 -.00021342772819526987
20111 "KR7000030007" "Woori Bank"      2015 5   .009928377197334692    .007918297178280916
20114 "KR7000030007" "Woori Bank"      2015 5  .0032876741941917083 -.00021178944076345908
20115 "KR7000030007" "Woori Bank"      2015 5   -.02548614398334065    .008600699094443035
20116 "KR7000030007" "Woori Bank"      2015 5  -.014697833835692696    .004690885918358176
20117 "KR7000030007" "Woori Bank"      2015 5   .005678606968843293   -.005397958207527501
20118 "KR7000030007" "Woori Bank"      2015 5  -.003403293131707807  -.0009024993689995528
20121 "KR7000030007" "Woori Bank"      2015 5   .009049835519917856   .0017529746104667774
20122 "KR7000030007" "Woori Bank"      2015 5   .046212843155131626 -.00036879200549115923
20123 "KR7000030007" "Woori Bank"      2015 5    .02127739844728488    .005532934480386606
20124 "KR7000030007" "Woori Bank"      2015 5  -.040821994520255166   -.005082207162778657
20125 "KR7000030007" "Woori Bank"      2015 5  .0021905813798186818   .0013714194291433337
20128 "KR7000030007" "Woori Bank"      2015 5  .0043668191663403895   -.004366416334350988
20129 "KR7000030007" "Woori Bank"      2015 5  -.002181025946360226  -.0057380542000511235
20130 "KR7000030007" "Woori Bank"      2015 5   .001091107582334557    .005070137305181908
20131 "KR7000030007" "Woori Bank"      2015 5  -.007662872745569137  -.0020939830100175924
20132 "KR7000030007" "Woori Bank"      2015 5    .02710192977779118     .00814032239764542
20135 "KR7000030007" "Woori Bank"      2015 5  .0010689471889049637  .00037285512967290843
20136 "KR7000030007" "Woori Bank"      2015 5   .015898586067798204   .0016429916341868754
20142 "KR7000030007" "Woori Bank"      2015 5   .004197277935339916    .003531954081862487
20143 "KR7000030007" "Woori Bank"      2015 5                     0    .003919376498156303
20144 "KR7000030007" "Woori Bank"      2015 5   .006263068489562273     .00723546552790485
20145 "KR7000030007" "Woori Bank"      2015 5                     0   .0013103891621661812
20146 "KR7000030007" "Woori Bank"      2015 5  -.013619906715270726   -.003659325299414023
20149 "KR7000030007" "Woori Bank"      2015 5   -.01916991610772017   .0055290515758953255
20150 "KR7000030007" "Woori Bank"      2015 5    .02127739844728488   .0022860354261399285
20151 "KR7000030007" "Woori Bank"      2015 5  -.006335818449085783  -.0015451277804222922
20152 "KR7000030007" "Woori Bank"      2015 5  -.003183026560121101  .00004503749372110945
20153 "KR7000030007" "Woori Bank"      2015 5  -.010683862308312577    .007259487621433927
20156 "KR7000030007" "Woori Bank"      2015 5 -.0010746911297654092   -.010045618908584441
20157 "KR7000030007" "Woori Bank"      2015 5  -.007555351644449502   -.004047682637351913
20158 "KR7000030007" "Woori Bank"      2015 5    -.0097986630487021   -.001987089618939057
20159 "KR7000030007" "Woori Bank"      2015 5    .03335435603959268   -.005182958492457466
20160 "KR7000030007" "Woori Bank"      2015 5  .0021141656923355564    .007683829554033234
20163 "KR7000030007" "Woori Bank"      2015 5   .023996978311350333   .0007752094460382831
20164 "KR7000030007" "Woori Bank"      2015 5     .0404095383378767    .021199427798639255
20165 "KR7000030007" "Woori Bank"      2015 5  -.004962789342129014  -.0007195024901609938
20166 "KR7000030007" "Woori Bank"      2015 5  -.011005613836602129    .004643004253727621
20167 "KR7000030007" "Woori Bank"      2015 5  -.018274620243481556 -.00031900823066373645
20170 "KR7000030007" "Woori Bank"      2015 5  -.015488177442800026  -.0003191100293904606
20171 "KR7000030007" "Woori Bank"      2015 5    -.0178482428247918   .0023443104832597387
20172 "KR7000030007" "Woori Bank"      2015 5  -.024666129890193834   .0007051599389205643
20173 "KR7000030007" "Woori Bank"      2015 5   .011872778430284345   -.009962275256084613
20174 "KR7000030007" "Woori Bank"      2015 5    .00748266452267182  -.0013655391547909325
20177 "KR7000030007" "Woori Bank"      2015 5    -.0085562019311958    .005057000682744104
20179 "KR7000030007" "Woori Bank"      2015 5  -.004305711711211868   -.000783542687722157
20180 "KR7000030007" "Woori Bank"      2015 5  -.006493529310548343  .00030560539680647816
20181 "KR7000030007" "Woori Bank"      2015 5    .00972454989199478    .008025587378348024
20184 "KR7000030007" "Woori Bank"      2015 5   -.01735401469315162   .0004936642450294423
20185 "KR7000030007" "Woori Bank"      2015 5   .006543098588935887  .00029315053978637364
20186 "KR7000030007" "Woori Bank"      2015 5   .014031530804257916    .005956732799061367
20187 "KR7000030007" "Woori Bank"      2015 5  -.002145923570276771  -.0001894063573351822
20188 "KR7000030007" "Woori Bank"      2015 5   .006424004961355101    .013934432278219892
20191 "KR7000030007" "Woori Bank"      2015 5     .0106158109476561    .005331205839576944
20192 "KR7000030007" "Woori Bank"      2015 5    .13141722693218716    .006079854572661524
20193 "KR7000030007" "Woori Bank"      2015 5  -.037740327982847086   .0038944392838946777
20194 "KR7000030007" "Woori Bank"      2015 5    .02843793532053341    .009361878388573858
20195 "KR7000030007" "Woori Bank"      2015 5  -.018868484304382805   .0016809080859297193
20198 "KR7000030007" "Woori Bank"      2015 5    .03278982282299097    .001496430523920293
20199 "KR7000030007" "Woori Bank"      2015 5   -.01392133851860812  -.0008947920904599881
20200 "KR7000030007" "Woori Bank"      2015 5  -.028437935320533514 -.00041970956714499925
20201 "KR7000030007" "Woori Bank"      2015 5    .05608946665104358     .01367542679967549
20202 "KR7000030007" "Woori Bank"      2015 5   .018018505502678212   -.006281737905363044
20205 "KR7000030007" "Woori Bank"      2015 5                     0  -.0010469410361146842
20206 "KR7000030007" "Woori Bank"      2015 5   -.04567003683318832   -.004585150221200382
20207 "KR7000030007" "Woori Bank"      2015 5    .00930239266231341  -.0023494871307076426
20208 "KR7000030007" "Woori Bank"      2015 5  -.004640379556502225    -.00724158862753357
20212 "KR7000030007" "Woori Bank"      2015 5    .04100802372737704    .002375922608964443
20214 "KR7000030007" "Woori Bank"      2015 5  -.013483350337286988   -.013052458123952492
20215 "KR7000030007" "Woori Bank"      2015 5   -.03218668649590133  -.0064735018394815325
20216 "KR7000030007" "Woori Bank"      2015 5  -.004683849312426314  -.0026241958112451346
20219 "KR7000030007" "Woori Bank"      2015 5  .0046838493124264375    .005670722121927975
20220 "KR7000030007" "Woori Bank"      2015 5   .023095714794649395  -.0002908813486886191
20221 "KR7000030007" "Woori Bank"      2015 5   .009090971701252105    .008259505079021973
20222 "KR7000030007" "Woori Bank"      2015 5  -.027524673390090033    .002914166459947342
20223 "KR7000030007" "Woori Bank"      2015 5  -.009345862418237658  -.0065439345105259205
20226 "KR7000030007" "Woori Bank"      2015 5  -.023754086008107144   .0034216259076661018
20227 "KR7000030007" "Woori Bank"      2015 5  .0047961722634930135     .00336752337922178
20228 "KR7000030007" "Woori Bank"      2015 5  .0047732787526575905    .008773900933144374
20229 "KR7000030007" "Woori Bank"      2015 5  -.019231361927887644   -.007850169937084888
20230 "KR7000030007" "Woori Bank"      2015 5    .03341599691984402    .010911558725417332
20234 "KR7000030007" "Woori Bank"      2015 5  -.004705891037412617  -.0012122343894042287
20235 "KR7000030007" "Woori Bank"      2015 5  -.019048194970694474   -.016937596162021312
20236 "KR7000030007" "Woori Bank"      2015 5                     0   .0016072486089562371
20237 "KR7000030007" "Woori Bank"      2015 5   .009569451016150673   .0018505858763285005
20240 "KR7000030007" "Woori Bank"      2015 5  -.009569451016150673   -.005894965579205508
20241 "KR7000030007" "Woori Bank"      2015 5  -.009661910911736859   -.011351446091190737
20242 "KR7000030007" "Woori Bank"      2015 5   .004842624475787991   -.007475045671142922
20243 "KR7000030007" "Woori Bank"      2015 5  -.019512814223581715    .004690508161966302
20244 "KR7000030007" "Woori Bank"      2015 5  -.017893121514049332   -.002298984823091618
20247 "KR7000030007" "Woori Bank"      2015 5    .00799205053133787  -.0014080795108989419
20248 "KR7000030007" "Woori Bank"      2015 5   .004962789342129097  -.0005618494681876902
20249 "KR7000030007" "Woori Bank"      2015 5   .014742281737203431   -.006176894024294554
20250 "KR7000030007" "Woori Bank"      2015 5  -.014742281737203431    .002575507822262427
20251 "KR7000030007" "Woori Bank"      2015 5  -.004962789342129014   -.002161226314493689
20254 "KR7000030007" "Woori Bank"      2015 5    -.0170601227453083   -.004811353307280131
20255 "KR7000030007" "Woori Bank"      2015 5  -.025629285949742304   -.006681364266956086
20256 "KR7000030007" "Woori Bank"      2015 5   .008273056493199283   .0030219681525592203
20257 "KR7000030007" "Woori Bank"      2015 5   -.01661512781059464   .0034439316407167573
20258 "KR7000030007" "Woori Bank"      2015 5 -.0010476690324436888   .0024848135969057885
20261 "KR7000030007" "Woori Bank"      2015 5    .01353482400500776    .003997938101304466
20262 "KR7000030007" "Woori Bank"      2015 5    .03854432503988185    .012590946635880191
20263 "KR7000030007" "Woori Bank"      2015 5   .019705071079332337    .002078369156946911
20264 "KR7000030007" "Woori Bank"      2015 5                     0 -.00022538777582540535
20265 "KR7000030007" "Woori Bank"      2015 5  -.009804000096620856      .0024908283381862
20268 "KR7000030007" "Woori Bank"      2015 5   -.04225980928988261   -.014344641559333871
20269 "KR7000030007" "Woori Bank"      2015 5   .007168489478612497    .006631718579511737
16072 "KR7000040006" "HyosungMtr&Mach" 2004 6                     .                      .
16075 "KR7000040006" "HyosungMtr&Mach" 2004 6  -.009756174945364685   .0034521354845295413
16076 "KR7000040006" "HyosungMtr&Mach" 2004 6   -.05026183478088831   -.000813338800428396
16077 "KR7000040006" "HyosungMtr&Mach" 2004 6    .08872811560868422    .004410791761737322
16078 "KR7000040006" "HyosungMtr&Mach" 2004 6   -.05826890812397576   -.003536782553281773
16079 "KR7000040006" "HyosungMtr&Mach" 2004 6   -.01005033585350145     .02530355076926674
16082 "KR7000040006" "HyosungMtr&Mach" 2004 6                     0    .006509226666746321
16083 "KR7000040006" "HyosungMtr&Mach" 2004 6    .02985296314968113   -.002777746869099685
16084 "KR7000040006" "HyosungMtr&Mach" 2004 6   -.02985296314968116   .0014016079559652714
16085 "KR7000040006" "HyosungMtr&Mach" 2004 6                     0  -.0046718031269089725
end
format %tdNN/DD/CCYY date

Unfortunately this code does not provide a desired outcome.

Where I went wrong?

Thanks a lot in advance.

[Nick Cox]

1. You saved a scalar result. statsby can save you the coefficients, but you need the original dataset to compute residuals.

2. A bigger deal is that date IssueCode id identify single observations, so you are trying to fit a line to each individual data point, which makes no sense.

[Olena Onishchenko]

Thanks Nick. I see.
Then I honestly do not know how to implemet my task in Stata.

All I need is to run a univariate regression on my data (reg RTN KOSPI_RTN) and collect residuals for each observation.

What code should I use ?

Thanks a lot in advance.

[Clyde Schechter]

If you just need the residuals from a single regression on all the data, then it's pretty simple:

reg RTN KOSPI_RTRN
predict resid, resid

If the regression needs to be done separately for each issue code (I give this as an example because I don't understand what your needs really are here) then it's a little more complicated:

gen resid = .
levelsof IssueCode, local(IssueCodes)
foreach i lof local IssueCodes {
     regress RTN KOSPI_RTRN if IssueCode == `"`i'"'
     predict temp, resid
     replace resid = temp if IssueCode == `"`i'"'
     drop temp
}

You can modify this code to accommodate whatever variable(s) define the observations to be used in each regression.

[Olena Onishchenko]

Thank you Clyde. The loop you suggested is what I was looking for.