Hello everyone. I'm not coming for help this time, I want share something with you instead. I've been testing the forecast capability of a time serie. For that, I builded a pseudo-out-of-sample (POOS) model and I used the Root Mean Squared Forecast Error (RMSFE) as a measure of forecast accuracy.
Following Stock & Watson, I used a Autoregressive Distributed Lag (ADL) Model with lags from the dependent and independent variable. Y is the dependent varible and X the independent.
My sample goes in quarters from 1996q1 to 2016q1. I divided the sample in two; the estimation subsample and the forecasting subsample. As I have very few data, I did n=2T/3, but with enought data you can just divided in halfs. In this way, I do the forecasting from 2010q1 to 2016q1. The advantages of POOS models is that you can test the forecasting accurancy without the need of waiting for the real data to come, so you can compare the forecast with the observed data and calculate the RMSFE.
So here is the algorithm that I used.
***First, find the optimum lag number
forvalues i = 1/20 {
forvalues j = 1/20 {
*di `i' `j'
qui gen f_Y_X1`i'_`j' = .
qui gen stdf_Y_X1`i'_`j' = .
qui gen sqdiff1`i'_`j' = .
***pseudo-out-of-sample (POOS) model to estimate
forvalues p = `=tq(2010q1)'/`=tq(2016q1)' {
qui regress Y L(1/`i').Y L(1/`j').X if quarter<`p'
qui predict Temp_Y_hat, xb
qui predict Temp_Y_rmsfe, stdf
qui replace f_Y_X1`i'_`j' = Temp_Y_hat if quarter==`p'+1
qui replace stdf_Y_X1`i'_`j' = Temp_Y_rmsfe if quarter==`p'+1
qui replace sqdiff1`i'_`j' = (Y - Temp_Y_hat)^2 if quarter==`p'+1
qui drop Temp_Y_hat Temp_Y_rmsfe
}
***compute the information criteria for the i lag
qui estat ic
mat IC = r(S)
scalar BIC`i'_`j' = el(IC, 1, 6)
scalar AIC`i'_`j' = el(IC, 1, 5)
***compute the RMSFE for the i lag
qui egen meansqdiff`i'_`j' = mean(sqdiff1`i'_`j')
qui scalar RMSFE`i'_`j' = sqrt(meansqdiff`i'_`j')
qui drop meansqdiff`i'_`j'
}
}
***find the best criteria between all lags
scalar minBIC = .
scalar minAIC = .
scalar minRMSFE = .
forvalues i = 1/20 {
forvalues j =1/20 {
if BIC`i'_`j' < minBIC {
scalar minBIC = BIC`i'_`j'
local minBICij ",`i' , `j'"
}
if AIC`i'_`j' < minAIC {
scalar minAIC = AIC`i'_`j'
local minAICij ",`i' , `j'"
}
if RMSFE`i'_`j' < minRMSFE {
scalar minRMSFE = RMSFE`i'_`j'
local minRMSFEij ",`i' , `j'"
}
}
}
di "min BIC lag"`minBICij' " : " minBIC
di "min AIC lag"`minAICij' " : " minAIC
di "min RMSFE lag"`minRMSFEij' " : " minRMSFE
*****
*** Now estimate the POOS model with the values found above. `p'+1 is for one period ahead and you can add more period changing the "1".
*****
qui gen f_Y_X1 = .
qui gen stdf_Y_Y1 = .
qui gen sqdiff1 = .
forvalues p = `=tq(2010q1)'/`=tq(2016q1)' {
qui regress Y L(1/11).Y L(1/1).X if quarter<`p'
qui predict Temp_Y_hat, xb
qui predict Temp_X_rmsfe, stdf
qui replace f_Y_X1 = Temp_Y_hat if quarter==`p'+1
qui replace stdf_Y_X1 = Temp_Y_rmsfe if quarter==`p'+1
qui replace sqdiff1 = (Y - Temp_Y_hat)^2 if quarter==`p'+1
qui drop Temp_Y_hat Temp_Y_rmsfe
}
***Compute the RMSFE for the POOS estimated model
qui egen meansqdiff1 = mean(sqdiff1)
qui scalar RMSFE_Y_X1 = sqrt(meansqdiff1)
di "RMSFE of Y and X 1 periods ahead =" %9.4f scalar(RMSFE_Y_X1)
***Compute the confidence intervals
qui gen fcastHigh = f_Y_X1 + 1.96*stdf_Y_X1
qui gen fcastLow = f_Y_X1 - 1.96*stdf_Y_X1
***graph
qui twoway (rarea fcastLow fcastHigh quarter, bcolor(gs14) legend(lab(1 "IC 95%"))) (tsline Y f_Y_X1, lpattern(dash solid) legend(lab(2 "Observed Y") lab(3 "Forecasted Y"))), title("Pseudo-out-of-sample forecasting") subtitle("Y and X" "1 period ahead") ytitle("Var (%)") xtitle("Quarter") xlabel(#10, angle(45)) xmticks(#40) note("RMSFE of Y 1 period ahead = `=scalar(RMSFE_Y_X1)'") name(f_YX1_graph, replace)
qui drop meansqdiff1 fcastHigh fcastLow
qui scalar drop _all
I hope this works for you.
If you find and error or something that doesn´t work, plase let me know.
Greets!
Following Stock & Watson, I used a Autoregressive Distributed Lag (ADL) Model with lags from the dependent and independent variable. Y is the dependent varible and X the independent.
My sample goes in quarters from 1996q1 to 2016q1. I divided the sample in two; the estimation subsample and the forecasting subsample. As I have very few data, I did n=2T/3, but with enought data you can just divided in halfs. In this way, I do the forecasting from 2010q1 to 2016q1. The advantages of POOS models is that you can test the forecasting accurancy without the need of waiting for the real data to come, so you can compare the forecast with the observed data and calculate the RMSFE.
So here is the algorithm that I used.
***First, find the optimum lag number
forvalues i = 1/20 {
forvalues j = 1/20 {
*di `i' `j'
qui gen f_Y_X1`i'_`j' = .
qui gen stdf_Y_X1`i'_`j' = .
qui gen sqdiff1`i'_`j' = .
***pseudo-out-of-sample (POOS) model to estimate
forvalues p = `=tq(2010q1)'/`=tq(2016q1)' {
qui regress Y L(1/`i').Y L(1/`j').X if quarter<`p'
qui predict Temp_Y_hat, xb
qui predict Temp_Y_rmsfe, stdf
qui replace f_Y_X1`i'_`j' = Temp_Y_hat if quarter==`p'+1
qui replace stdf_Y_X1`i'_`j' = Temp_Y_rmsfe if quarter==`p'+1
qui replace sqdiff1`i'_`j' = (Y - Temp_Y_hat)^2 if quarter==`p'+1
qui drop Temp_Y_hat Temp_Y_rmsfe
}
***compute the information criteria for the i lag
qui estat ic
mat IC = r(S)
scalar BIC`i'_`j' = el(IC, 1, 6)
scalar AIC`i'_`j' = el(IC, 1, 5)
***compute the RMSFE for the i lag
qui egen meansqdiff`i'_`j' = mean(sqdiff1`i'_`j')
qui scalar RMSFE`i'_`j' = sqrt(meansqdiff`i'_`j')
qui drop meansqdiff`i'_`j'
}
}
***find the best criteria between all lags
scalar minBIC = .
scalar minAIC = .
scalar minRMSFE = .
forvalues i = 1/20 {
forvalues j =1/20 {
if BIC`i'_`j' < minBIC {
scalar minBIC = BIC`i'_`j'
local minBICij ",`i' , `j'"
}
if AIC`i'_`j' < minAIC {
scalar minAIC = AIC`i'_`j'
local minAICij ",`i' , `j'"
}
if RMSFE`i'_`j' < minRMSFE {
scalar minRMSFE = RMSFE`i'_`j'
local minRMSFEij ",`i' , `j'"
}
}
}
di "min BIC lag"`minBICij' " : " minBIC
di "min AIC lag"`minAICij' " : " minAIC
di "min RMSFE lag"`minRMSFEij' " : " minRMSFE
*****
*** Now estimate the POOS model with the values found above. `p'+1 is for one period ahead and you can add more period changing the "1".
*****
qui gen f_Y_X1 = .
qui gen stdf_Y_Y1 = .
qui gen sqdiff1 = .
forvalues p = `=tq(2010q1)'/`=tq(2016q1)' {
qui regress Y L(1/11).Y L(1/1).X if quarter<`p'
qui predict Temp_Y_hat, xb
qui predict Temp_X_rmsfe, stdf
qui replace f_Y_X1 = Temp_Y_hat if quarter==`p'+1
qui replace stdf_Y_X1 = Temp_Y_rmsfe if quarter==`p'+1
qui replace sqdiff1 = (Y - Temp_Y_hat)^2 if quarter==`p'+1
qui drop Temp_Y_hat Temp_Y_rmsfe
}
***Compute the RMSFE for the POOS estimated model
qui egen meansqdiff1 = mean(sqdiff1)
qui scalar RMSFE_Y_X1 = sqrt(meansqdiff1)
di "RMSFE of Y and X 1 periods ahead =" %9.4f scalar(RMSFE_Y_X1)
***Compute the confidence intervals
qui gen fcastHigh = f_Y_X1 + 1.96*stdf_Y_X1
qui gen fcastLow = f_Y_X1 - 1.96*stdf_Y_X1
***graph
qui twoway (rarea fcastLow fcastHigh quarter, bcolor(gs14) legend(lab(1 "IC 95%"))) (tsline Y f_Y_X1, lpattern(dash solid) legend(lab(2 "Observed Y") lab(3 "Forecasted Y"))), title("Pseudo-out-of-sample forecasting") subtitle("Y and X" "1 period ahead") ytitle("Var (%)") xtitle("Quarter") xlabel(#10, angle(45)) xmticks(#40) note("RMSFE of Y 1 period ahead = `=scalar(RMSFE_Y_X1)'") name(f_YX1_graph, replace)
qui drop meansqdiff1 fcastHigh fcastLow
qui scalar drop _all
I hope this works for you.
If you find and error or something that doesn´t work, plase let me know.
Greets!
