Factor variables in the vselect command

[Federico Tedeschi]

Dear Stata listers,

I'm using Stata version 13.1.

I'd have a question about Stata user-written command "vselect", in particular about the inclusion of categorical variables.

It seems to me that, in case groups are more than 2, it is possible to tell the software they are factor variables in the usual way (with the prefix "i." before the variable and "xi" before the command), but not to consider all categories jointly (so that a given factor variable is either included or excluded in its whole by each regression).

Let me explain my point: in a stepwise regression, where var1 is the outcome and var2 and var3 the categorical regressors, it is possible to introduce the constraint that (in case of inclusion) all categories of var3 have to be considered jointly (of course, apart from the reference category) by using parentheses (while still considering all var2 categories different from the reference one as separated dummy variables). The command could be like that:

xi: sw, pe (0.05) pr (0.10): regress outcome i.var2 (i.var3)

On the contrary, it seems to me the inclusion of parentheses for regressors with the "vselect" command is useless.

E.g.:

xi:vselect outcome i.var2 (i.var3), best

and

xi:vselect outcome i.var2 i.var3, best

would be exactly treated in the same way (each category different from the reference one being treated as a separate variable indeed).

In my opinion, this makes results dependent on the reference category chosen for factor variables with more than 2 categories. Also, the possibility to force factor variables to be either completely included or excluded would reduce the numbers of options to consider for each possible number of regressors.

I was wondering whether there is some obvious solution to it by using some other commands in advance to define a list of variables (like "selectvars" or "tuples") and whether allowing for the inclusion of factor variables in the "fix" option was seen as a workaround for this problem (the inclusion of factor variables being decided outside the "vselect" command, and then the factor variables included being forced into each model by "fix").

Could you help me with this issue?

Thanks in advance.


Reference:

Lindsey C. and S. Sheather (2010), Variable Selection in Linear
Regression, The Stata Journal, 10:650-669.

[Richard Williams]

You may have to take that up with the authors. I will note that the program does not seem to be unique in that respect. The commands nestreg and stepwise do not work with regular factor variable notation. I am not sure if that is because of strong religious objections on Stata Corps part or if it is because nobody has ever bothered to program it, but in any event it can't be done. You would need to use xi: or create the dummies yourself.

[Nick Cox]

I don't know anything about the details here, but I suspect that the double status

1. It's a factor variable.

2. It's a bundle of indicator variables

just entails too much accountancy in some problems.

[Federico Tedeschi]

Thank you both for your replies.

You may have to take that up with the authors.

I wrote to both a few days ago both haven’t received any reply so far.

I will note that the program does not seem to be unique in that respect. The commands nestreg and stepwise do not work with regular factor variable notation. I am not sure if that is because of strong religious objections on Stata Corps part or if it is because nobody has ever bothered to program it, but in any event it can't be done. You would need to use xi: or create the dummies yourself.

With stepwise I have no problems: if I don’t use parentheses Stata considers all the dummy variables; if I do use them, I have the whole factor variable considered for removal/addiction at each step. My problem is not to create dummies, but the other way round: to have factor variables treated as single variables.

I don't know anything about the details here, but I suspect that the double status

1. It's a factor variable.

2. It's a bundle of indicator variables

just entails too much accountancy in some problems.

I’d like to tell the program to consider a factor variable as it is (your point 1) but basically vselect just considers it as a group of n-1 indicator variables (with n as the number of groups: your point 2). Are there commands that can perform point 1, apart from stepwise (that could miss the best subsets, depending on the chosen procedure) and nestreg (that would not allow me to compare non-nested subsets)?

[Rich Goldstein]

you may be able to add it yourself; first, make a copy of the file and only work with that, then, look at the following FAQ: http://www.stata.com/support/faqs/pr...ort/index.html

[Federico Tedeschi]

Thank you!

I think I could try to overcome the limitation of results being dependent on the reference category chosen, in this way. However, this also made me realize that the problem with vselect is not specific about factor variables, but relates to forcing 2 or more variables to be either all included or all excluded from the regression. I will ask a new question then.

[Richard Williams]

stepwise.ado includes lines like

gettoken term termlist : termlist, bind match(par)

Looking at -help gettoken- makes me think that the bind and/or match options may be necessary to do what you want. But I am just guessing here.

[Federico Tedeschi]

stepwise.ado includes lines like

gettoken term termlist : termlist, bind match(par)

Thank you. I guess this is the answer: the stepwise command gives sense to parentheses thanks to the lines you quote in the .ado file, while something like that is not there in the .ado file of vselect.

[Federico Tedeschi]

Dear Statalisters,

I don't know anything about the details here, but I suspect that the double status

1. It's a factor variable.

2. It's a bundle of indicator variables

just entails too much accountancy in some problems.

I've reconsidered such point to come out with what seemed to me a very simple solution: to create dummy variables for each category (with the "tabulate ..., generate ..." command).

I was wondering how the command would react to the "dummy variable trap".

I actually found something weird. I prepared two examples (on the same dataset) with just one and two variables (respectively) to make the issue simpler.

************************************************** ************************************************** ********************************************

1. Only stc:

xi: vselect comp_sat_imp stc_ind1-stc_ind3, best
11 observations containing missing predictor values


Response : comp_sat_imp
Selected predictors: stc_ind3 stc_ind2 stc_ind1

Optimal models:

# Preds R2ADJ C AIC AICC BIC
1 .0086653 2.813701 2789.95 2790.009 2797.992
2 .0062415 4.813701 2791.95 2792.048 2804.013
3 .0106288 4 2791.118 2791.266 2807.202

predictors for each model:

1 : stc_ind3
2 : stc_ind3 stc_ind1
3 : stc_ind3 stc_ind2 stc_ind1

. regress comp_sat_imp stc_ind1-stc_ind3
note: stc_ind1 omitted because of collinearity

Source | SS df MS Number of obs = 412
-------------+------------------------------ F( 2, 409) = 3.72
Model | 376.371394 2 188.185697 Prob > F = 0.0251
Residual | 20708.2558 409 50.6314323 R-squared = 0.0179
-------------+------------------------------ Adj R-squared = 0.0130
Total | 21084.6272 411 51.3007961 Root MSE = 7.1156

------------------------------------------------------------------------------
comp_sat_imp | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
stc_ind1 | 0 (omitted)
stc_ind2 | 1.333031 .7937243 1.68 0.094 -.2272573 2.893319
stc_ind3 | 3.396317 1.278511 2.66 0.008 .8830446 5.90959 _cons | 31.48237 .6550419 48.06 0.000 30.1947 32.77004

1. Stc and gender

xi: vselect comp_sat_imp stc_ind1-stc_ind3 gender, best
13 observations containing missing predictor values


Response : comp_sat_imp
Selected predictors: stc_ind1 stc_ind2 gender stc_ind3

Optimal models:

# Preds R2ADJ C AIC AICC BIC
1 .0080096 2.539321 2777.067 2777.126 2785.099
2 .0140283 1.065319 2775.566 2775.664 2787.614
3 .0115998 3.065319 2777.566 2777.714 2793.63
4 .0093191 5 2779.499 2779.708 2799.58

predictors for each model:

1 : stc_ind1
2 : stc_ind1 stc_ind2
3 : stc_ind1 stc_ind2 stc_ind3
4 : stc_ind1 stc_ind2 gender stc_ind3

************************************************** ************************************************** ******************************************

As you can see, the best model in terms of Adj R^2 in the former case (but in my real case with many variables it's also in terms of AIC and AICC) is the one including all the three dummy variables for the categorical variable "stc". You can also see that, when running the regression, one category is omitted "because of collinearity": the model wouldn't be identified with all parameters. Do you have an explanation for that behaviour of "vselect"?

In the latter example, I included "gender" among regressors, and the best choice with 3 parameters was to include all those of "stc": how is it possible to prefer to add a parameter that will be excluded by the regression due to underidentification instead of a "really new" one? Does this happen because the command takes into account the omission of redundant parameters? By the way, why is the Adj R^2 calculated by vselect different from the one proposed by Stata when using "regress"? Is a different criterion used?

I also considered the unrealistic hypothesis the command doesn't include the intercept: it's not the case, given with my real case the "dummy variable trap" occurs for two variables in the best model.