Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • idiosyncratic volatility

    Using below data, I want to calculate monthly idiosyncratic volatility of the stocks. For each month, idiosyncratic volatility is the standard deviation of the residuals obtaining by regressing within month daily returns (rt) on mkt, smb and hml. The code should run free of any error due to missing data on account of (rt). The code should generate for each stock month wise number of (rt) observations used for calculation of idiosyncratic volatility so that any estimate of idiosyncratic volatility that uses <17 (rt) observation are replaced with a missing observation value.

    Code:
    * Example generated by -dataex-. To install: ssc install dataex
    clear
    input float s_id str52 stock float(ddate rt mkt smb hml) byte month int year
    1 "3M India Ltd." 15522  .0070239  .0150011  .0010725  .0250334  7 2002
    1 "3M India Ltd." 15523 -.0076514 -.0018003  .0109898 -.0038538  7 2002
    1 "3M India Ltd." 15524  .0347495  .0030851   .004934 -.0112142  7 2002
    1 "3M India Ltd." 15525  .0998433   .005215  .0084408  .0050568  7 2002
    1 "3M India Ltd." 15526  -.019263  .0023367  .0151751  .0140474  7 2002
    1 "3M India Ltd." 15529 -.0010992  .0063464  .0131835  .0245329  7 2002
    1 "3M India Ltd." 15530  .0174534 -.0000992 -.0042045 -.0065735  7 2002
    1 "3M India Ltd." 15531 -.0316749 -.0091766 -.0110201 -.0138393  7 2002
    1 "3M India Ltd." 15532 -.0365298 -.0139226 -.0129029 -.0004927  7 2002
    1 "3M India Ltd." 15533 -.0011479  .0070776 -.0021916  .0038304  7 2002
    1 "3M India Ltd." 15536  -.022362 -.0027147 -.0173262 -.0280195  7 2002
    1 "3M India Ltd." 15537  .0167946 -.0174363  .0085582 -.0020982  7 2002
    1 "3M India Ltd." 15538 -.0268245 -.0076276 -.0116401 -.0118697  7 2002
    1 "3M India Ltd." 15539  .0130271  .0139086  .0077276  .0043189  7 2002
    1 "3M India Ltd." 15540  .0256999 -.0063961  .0045085  -.006089  7 2002
    1 "3M India Ltd." 15543 -.0413439 -.0267305 -.0102748 -.0299182  7 2002
    1 "3M India Ltd." 15544  .0402202  .0136311 -.0041524  .0035656  7 2002
    1 "3M India Ltd." 15545 -.0110579 -.0186549 -.0006778 -.0117335  7 2002
    1 "3M India Ltd." 15546  .0050184 -.0027835  .0017711 -.0061291  7 2002
    1 "3M India Ltd." 15547   -.00348 -.0251109 -.0064012 -.0200478  7 2002
    1 "3M India Ltd." 15550  .0055089 -.0087455 -.0187168 -.0199446  7 2002
    1 "3M India Ltd." 15551 -.0057925 -.0164984  .0019774 -.0058959  7 2002
    1 "3M India Ltd." 15552 -.0001567   .001973 -.0107227  .0109384  7 2002
    1 "3M India Ltd." 15553 -.0001567  -.002633  .0062768  .0087674  8 2002
    1 "3M India Ltd." 15554    .00951  .0032521  -.005443 -.0031184  8 2002
    1 "3M India Ltd." 15557  .0026495  .0071178  .0065931  .0205862  8 2002
    1 "3M India Ltd." 15558  .0006663  .0045082  .0050579 -.0041473  8 2002
    1 "3M India Ltd." 15559  .0090538  .0006917  .0140094 -.0082834  8 2002
    1 "3M India Ltd." 15560  -.022321 -.0178573 -.0044087 -.0161001  8 2002
    1 "3M India Ltd." 15561  .0023433  .0083427 -.0010041 -.0059277  8 2002
    1 "3M India Ltd." 15564 -.0013205  .0079767 -.0012146 -.0014728  8 2002
    1 "3M India Ltd." 15565 -.0014883  .0068159  .0006019 -.0024454  8 2002
    1 "3M India Ltd." 15566 -.0001567 -.0013984 -.0014991 -.0079164  8 2002
    1 "3M India Ltd." 15568    .01351  .0078742 -.0036486 -.0101744  8 2002
    1 "3M India Ltd." 15571 -.0136391  .0024809 -.0070199 -.0055307  8 2002
    1 "3M India Ltd." 15572  .0098433  .0084088 -.0021846 -.0037939  8 2002
    1 "3M India Ltd." 15573  -.003622  .0016594  -.003373  .0011826  8 2002
    1 "3M India Ltd." 15574  .0003401   -.00755  .0046005  .0009838  8 2002
    1 "3M India Ltd." 15575  .0041465  .0062263 -.0014517 -.0113817  8 2002
    1 "3M India Ltd." 15578         .  .0038196 -.0009357 -.0050024  8 2002
    1 "3M India Ltd." 15579         . -.0106739 -.0020531  .0005911  8 2002
    1 "3M India Ltd." 15580  .0006641  -.000084 -.0053981 -.0016628  8 2002
    1 "3M India Ltd." 15581 -.0009746 -.0070769  .0026259 -.0131091  8 2002
    1 "3M India Ltd." 15582    .03591  .0159023 -.0070818 -.0079758  8 2002
    1 "3M India Ltd." 15585 -.0033202  .0018853  .0056424   .012331  9 2002
    1 "3M India Ltd." 15586 -.0049175  -.004796 -.0038374 -.0028805  9 2002
    1 "3M India Ltd." 15587  .0046312  .0042594  .0003182 -.0083954  9 2002
    1 "3M India Ltd." 15588 -.0001535  .0007766 -.0000877  .0066088  9 2002
    1 "3M India Ltd." 15589 -.0147566 -.0111536  .0005437  .0009158  9 2002
    1 "3M India Ltd." 15592 -.0127179 -.0157857  .0072087 -.0156531  9 2002
    1 "3M India Ltd." 15594  .0112647 -.0010726  .0084194 -.0020316  9 2002
    1 "3M India Ltd." 15595         .  .0037487  -.002439  .0042637  9 2002
    1 "3M India Ltd." 15596         . -.0071694  .0029471  .0155653  9 2002
    1 "3M India Ltd." 15599 -.0001545 -.0105595  .0048443 -.0018094  9 2002
    1 "3M India Ltd." 15600   .059039  .0070672 -.0031071 -.0149273  9 2002
    1 "3M India Ltd." 15601 -.0103581 -.0106953 -.0022621  .0017017  9 2002
    1 "3M India Ltd." 15602 -.0153864 -.0097248 -.0001367 -.0037652  9 2002
    1 "3M India Ltd." 15603 -.0099978 -.0008097 -.0013893  .0017761  9 2002
    1 "3M India Ltd." 15606 -.0050468 -.0019508 -.0000492 -.0052543  9 2002
    1 "3M India Ltd." 15607 -.0088761 -.0042396 -.0068216 -.0092406  9 2002
    1 "3M India Ltd." 15608  .0056025  .0025061  .0000314  -.006337  9 2002
    1 "3M India Ltd." 15609  .0073189 -.0004933  .0086188  .0001865  9 2002
    1 "3M India Ltd." 15610 -.0053648  .0055308  .0021786 -.0019964  9 2002
    1 "3M India Ltd." 15613  -.000473 -.0152616 -.0012197  .0102552  9 2002
    1 "3M India Ltd." 15614  -.012854 -.0085113 -.0021432 -.0043831 10 2002
    1 "3M India Ltd." 15616  .0127061 -.0026873 -.0023763  .0042414 10 2002
    1 "3M India Ltd." 15617 -.0096794 -.0010751 -.0022495 -.0021289 10 2002
    1 "3M India Ltd." 15620         .  .0136201 -.0063736  .0131132 10 2002
    1 "3M India Ltd." 15621         .  .0069649  .0020557 -.0023531 10 2002
    1 "3M India Ltd." 15622  .0104281  -.006531  .0052737 -.0015805 10 2002
    1 "3M India Ltd." 15623  -.000473  .0051172 -.0079882 -.0030432 10 2002
    1 "3M India Ltd." 15624 -.0001556  .0143486 -.0022615 -.0079703 10 2002
    1 "3M India Ltd." 15627 -.0001556 -.0017215  .0016141  .0010462 10 2002
    1 "3M India Ltd." 15629 -.0001545 -.0038666  .0031139 -.0019318 10 2002
    1 "3M India Ltd." 15630 -.0001545  .0014755 -.0026364 -.0031856 10 2002
    1 "3M India Ltd." 15631 -.0001545  .0041047   .001627  .0002395 10 2002
    1 "3M India Ltd." 15634 -.0033291 -.0043123 -.0007723   .002945 10 2002
    1 "3M India Ltd." 15635 -.0033393 -.0148559  .0035002   .001241 10 2002
    1 "3M India Ltd." 15636  .0004844 -.0055717 -.0080572  .0063877 10 2002
    1 "3M India Ltd." 15637 -.0103717 -.0096719  .0019403 -.0074317 10 2002
    1 "3M India Ltd." 15638  .0062971 -.0110375    -.0053  .0101476 10 2002
    1 "3M India Ltd." 15641 -.0065648 -.0106246  .0013932  .0061909 10 2002
    1 "3M India Ltd." 15642  .0159745  .0160544 -.0007758 -.0000434 10 2002
    1 "3M India Ltd." 15643         .  .0036266  -.005743  .0086952 10 2002
    1 "3M India Ltd." 15644         .  .0113656 -.0011714 -.0008506 10 2002
    1 "3M India Ltd." 15676  .0017762  .0210703  .0087037  .0161378 12 2002
    1 "3M India Ltd." 15677 -.0127179 -.0150408 -.0021614 -.0130539 12 2002
    1 "3M India Ltd." 15678 -.0003017 -.0132719  .0003208  .0000655 12 2002
    1 "3M India Ltd." 15679 -.0057856  .0085134  .0028393  .0036829 12 2002
    1 "3M India Ltd." 15680 -.0035468  .0257752 -.0179969  .0035259 12 2002
    1 "3M India Ltd." 15683 -.0061626 -.0169007  .0058077 -.0046917 12 2002
    1 "3M India Ltd." 15684  .0060819  .0056933 -.0015801 -.0127855 12 2002
    1 "3M India Ltd." 15685 -.0069818  .0027269 -.0054806  .0062998 12 2002
    1 "3M India Ltd." 15686  .0175493  .0051327 -.0062228 -.0039682 12 2002
    1 "3M India Ltd." 15687  -.001918  .0087157 -.0099294  .0073515 12 2002
    1 "3M India Ltd." 15690  .0272724  -.005858 -.0033856  .0020016 12 2002
    1 "3M India Ltd." 15691         . -.0048832 -.0019622 -.0088799 12 2002
    1 "3M India Ltd." 15692         .  .0049223 -.0009646  .0010079 12 2002
    1 "3M India Ltd." 15693 -.0206242 -.0022744  .0086702 -.0012528 12 2002
    1 "3M India Ltd." 15694 -.0035511  .0034494 -.0024596  .0046267 12 2002
    end
    format %td ddate
    Last edited by Sartaj Hussain; 23 Jul 2021, 13:41.

  • #2
    Code:
    capture program drop one_stock_month
    program define one_stock_month
        regress rt mkt smb hml
        predict resid, resid
        summ resid
        if `r(N)' >= 17 {
            gen idio_vol = r(sd)
            gen n_obs = e(N)
        }
        exit
    end
    
    gen mdate = ym(year, month)
    assert missing(mdate) == missing(month, year)
    format mdate %tm
    
    
    runby one_stock_month, by(stock mdate)
    -runby- is written by Robert Picard and me, and is available from SSC.

    I used the year and month variables to define the month in this code. You also have a variable ddate, and it appears to correspond to be consistent with that as well, at least in the example. But if those are not always consistent, and if the correct way to define month for this purpose is based on ddate, then the code will need modification accordingly. (-help mofd()-).

    Comment


    • #3
      Please check if idiosyncratic volatility measurement in #2 follows below procedure. The first two i feel is properly taken care by the #2. The third step needs a look. My apologies if not made clear in #1

      1. For each stock in each month, generate daily residuals with given regression equation.
      2. Calculate daily volatility (standard deviation) of residuals.
      3. Multiply this daily volatility with square root of no of observations in a month to get monthly estimate.

      Please update code for this change.

      Moreover, this is the summary of report given upon execution of the code #1. I wanted to know what does by-groups with errors = 135 stand for. Is it worth concern.

      Number of by-groups = 7,236
      by-groups with errors = 135
      by-groups with no data = 0
      Observations processed = 149,544
      Observations saved = 146,770

      And, lastly code #1 does not generate missing observation estimates for idio_vol if rt for a given month is missing. This was needed to keep length of idio_vol variable intact so that it is easy to synchronise with other variables in a cross section.
      Last edited by Sartaj Hussain; 24 Jul 2021, 07:20.

      Comment


      • #4
        The code given in #2 fulfills everything you asked for in #1. You are now changing your request. Criterion 3 in #3 is new. As for "And, lastly code #1 does not generate missing observation estimates for idio_vol if rt for a given month is missing," that, too, is a new request. Both of these, I believe, are well within your capability of implementing without assistance.

        "This was needed to keep length of idio_vol variable intact so that it is easy to synchronise with other variables in a cross section." I have no idea what this means.

        Concerning the by-groups with errors, you can find out the cause by adding the -verbose- option to the -runby- command. You will then get the output along the way, including error messages. However, from my experience with this kind of programming, my hunch is that the by-groups with errors constitute stock-month groups where there are too few observations to carry out the regression. On the assumption that this is the case, you can modify the code as follows:

        Code:
        capture program drop one_stock_month
        program define one_stock_month
            capture regress rt mkt smb hml
            if c(rc) == 0 {
                predict resid, resid
                summ resid
                if `r(N)' >= 17 {
                    gen idio_vol = r(sd)
                    gen n_obs = e(N)
                }
            }
            else if !inlist(c(rc), 2000, 2001) { // ERROR OTHER THAN INSUFFICIENT OBSERVATIONS
                gen unexpected_error_code = c(rc)
                assert 0
            }
            exit
        end
        
        gen mdate = ym(year, month)
        assert missing(mdate) == missing(month, year)
        format mdate %tm
        
        
        runby one_stock_month, by(stock mdate)
        This code will, in the event of a group with too few observations to carry out the regression, simply move on to the next group. It will not be counted as a by-group with an error. In the event of some other unexpected error arising, it will create a new variable in your data set containing the error code for the problem it encountered, and it will count it as a by-group with an error. If, with this code you get by-groups with errors, you will know that it is due to something other than insufficient observations, and you can look in the data set itself to find the error codes associated with the problems, which you can then investigate.

        Comment


        • #5
          Yes, I will try on getting missing observations for idio_vol when rt is missing. I think there is a command: 'tsfill' that can help. About criterion 3 in #3, is below line correct. If yes, where should it be inserted in code at #4

          gen idiovol_m = idio_vol * sqrt(n_obs)
          Last edited by Sartaj Hussain; 24 Jul 2021, 13:32.

          Comment


          • #6
            Yes, that's correct.

            Comment


            • #7
              Kudos! Well, where to place this line in the code #4

              Comment


              • #8
                I would put it at the very end, after -runby-.

                Comment

                Working...
                X