model selection criteria

[Silvia Papagni]

Hello,
I'm using Stata 12.1 for my dissertation and I have a problem with saving and using the information criterion which prevents me to get the results I need.

I'm using the command 'tuples' to run regressions between the dependent variable (excess returns) and all the possible combinations of 9 independent variables ( this gives me 511 regressions to run) using the first 120 monthly observations. Then I performed the one-step ahead prediction for month 121 for each model, but I actually need only the prediction given by the best model according to an information criterion.

For this reason I need to find the BIC for each regression and then choose the one which has the lowest value of the Information Criterion considered.
I created a vector containing all the 511 values of the BIC criterion and then I generated a variable Bicmin equal to the minimum value of the elements of the vector. Finally I generated a variable predmax that gives me the value of the prediction just for the model that minimizes the BIC criterion (gen predmax=prev121 if BIcvec1==BIcmin).

I know that these steps are fine becouse I applied exactly the same procedure using the Rsquare instead of the BIC criterion and I had no problems. When using the BIC at the last step the variable predmax gives me a vector with all the predictions and so it doesn't select the best one ( it seems that the if condition doesn't work).
The problem must be due to the use of the BIC (the same happens with the AIC). When I ask 'display bic1' it doesn't appear anything, whereas with the R(2) I got the value for the corresponding regression model. At the same time if I write ' display bic600' it says: doesn't found (since I have only 511 models) so it seems that these values are actually stored somewhere but the program couldn't use them in the next steps.

This is the code I applied:

tuples L2.indprod L2.M1 L2.CPI L.bond_ret L2.bond_ret L.bill_returns L2.bill_returns L.D_P L.E_P

forval i = 1/`ntuples' {

quietly regress ex_return `tuple`i'' if tin(1 , 120)
predict pred`i' in 121
estat ic
scalar bic`i'=r(BIC)

}


/* IN PREV121 all the 511 PREDICTIONS */

forval i = 1/`ntuples' {
gen C`i'=bic`i'


}

set matsize 600
mkmat C1-C511, mat(BIc)
matrix tBIc=BIc'
svmat double tBIc, name(BIcvec)
mkmat BIcvec1, matrix(mBIcvec)

egen BIcmin=min(BIcvec1)

mkmat pred1-pred511, mat(pred)
matrix tpred=pred'
svmat double tpred, name(prev)
mkmat prev1, matrix(previ)

gen predmax=prev121 if BIcvec1==BIcmin
list predmax


Whereas this is the code using the R(2) as model selection criterion:


tuples L2.indprod L2.M1 L2.CPI L.bond_ret L2.bond_ret L.bill_returns L2.bill_returns L.D_P L.E_P

forval i = 1/`ntuples' {

quietly regress ex_return `tuple`i'' if tin(1 , 120)
scalar r2_`i'= e(r2)

predict pred`i' in 121
}

/* IN PREV121 there are all the 511 PREDICTIONS */

forval i = 1/`ntuples' {
gen A`i'=r2_`i'


}

set matsize 600
mkmat A1-A511, mat(Ar2)
matrix tAr2=Ar2'
svmat double tAr2, name(r2vec)
mkmat r2vec1, matrix(mr2vec)

egen r2max=max(r2vec1)

mkmat pred1-pred511, mat(pred)
matrix tpred=pred'
svmat double tpred, name(prev)
mkmat prev1, matrix(previ)

gen predmax=prev121 if r2vec1==r2max
list predmax

I can't understand what is the problem with the BIC/AIC becouse I applied the same steps.
I'm sorry if this may be a trivial question but I have been using stata not for so long so maybe I'm just missing some basic concept.
Any hints would be really welcome.
Thank you very much.

Kind regards,
Silvia

[Sebastian Kripfganz]

Without checking all the lines of your code, the problem seems to be at the following part:

estat ic
scalar bic`i'=r(BIC)

estat ic does not return a scalar stored in r(BIC). Your coding will therefore allocate a missing value to the scalar bic`i'. Instead, the following code will do the job:

estat ic
mat stats = r(S)
scalar bic`i' = el(stats, 1, 6)

Type help estat and scroll down to the very end to see that estat ic returns the results only in the matrix r(S). The BIC is then the entry in the first row and sixth column of this matrix.

[Silvia Papagni]

Thank you so much, everything works great now!