The "optimization" of lambda is carried out by trying out different values of lambda and retaining the results that give the lowest value of HRMSE among those values tried. But lambda is a continuous variable. The code shown considers as possible values of lambda 0.78, 0.79, 0.80, ..., 0.95, 0.96, 0.97. Values of lambda between those numbers are not considered. So when the calculation concludes, say, that 0.83 is the optimal value of lambda, we cannot exclude the possibility that the actual optimal value is .0833 or .08275169 or something like that, because those values were not considered. All we can say is that to within a precision of 0.01, 0.83 turned out best. If you need more precision on lambda than that, we would have to try a series of values that increased more slowly than 0.01 at each step. For example, in principle, it is possible to try lambda values of 0.780, 0.781, 0.782, 0.783, ..., 0.967, 0.968, 0.969, 0.097. This will give a value of lambda with a precision of 0.001. But it also involves 10 times as much calculation, which means the code runs 10 times slower. You can make this trade-off between precision and speed any way you like. (Including, you could go in the other direction: if a precision of 0.1 were sufficient, we could just do 0.7, 0.8, 0.9, and the result would be less precise but obtained in about 1/3 as much time.) The precision depends on just how fine-grained (difference between successive trial values of lambda) the analysis is.
The direct definition of sigma2 is the mean value of (rt - rt_mean)2. Given that the calculation of sigma2 is done many times in this code, using the -egen, sd()- function would make the code very slow. So I am using a recursive algorithm for calculating variances, Welford's algorithm. (Actually, a slight variation on Welford's algorithm that improves its numerical stability). I put in the comment naming it because, if you are not familiar with Welford's algorithm, you could look at it and have no idea what those lines of code are doing--it isn't immediately obvious. I wanted you to know that that was a calculation of the variance over the window. If you want to read more about different approaches to calculating variances, the Wikipedia article (https://en.wikipedia.org/wiki/Algori...ating_variance) is good. It does include the formulas for Welford's algorithm, and you will recognize that my code is an implementation of those.
The direct definition of sigma2 is the mean value of (rt - rt_mean)2. Given that the calculation of sigma2 is done many times in this code, using the -egen, sd()- function would make the code very slow. So I am using a recursive algorithm for calculating variances, Welford's algorithm. (Actually, a slight variation on Welford's algorithm that improves its numerical stability). I put in the comment naming it because, if you are not familiar with Welford's algorithm, you could look at it and have no idea what those lines of code are doing--it isn't immediately obvious. I wanted you to know that that was a calculation of the variance over the window. If you want to read more about different approaches to calculating variances, the Wikipedia article (https://en.wikipedia.org/wiki/Algori...ating_variance) is good. It does include the formulas for Welford's algorithm, and you will recognize that my code is an implementation of those.
Comment