Backward stepwise regression with interactions in meglm?

[Andrew Teer]

I have STATA 14. I'm using a mixed effects model and wanted to know the best approach to simplify our model. Right now, the model includes random effects of subjectID (there are 96 observations for each of the 33 subjects), 3 categorical variables (2 with two levels and 1 with three levels) and 2 continuous variables, and the dependent variable is continuous. Each subject undergoes the same task. The task has two epochs. Each epoch has 2 blocks. And each block is comprised of 8 trials of each of the 3 conditions in a fixed random order (24 trials per block). The syntax from what I have been running is this:

mixed c.dv epoch##block##condition##continuous_var1##continuo us_var2 || subjectID:

The problem I am having is that this model is quite large given all of the interaction terms, and I cannot get the stepwise command to work with interaction terms, which I expect to have some significance in this model.

Any advice would be appreciated. And if more any additional information would be helpful, please let me know.
Thank you!

[Clyde Schechter]

-stepwise- is one of the few parts of Stata's estimation apparatus that does not support factor variables. I have no inside knowledge of StataCorp's thinking, but I suspect that it is because they do not wish to continue supporting the -stepwise- command. It is little used and widely deprecated in the Stata community. For a short explanation of why stepwise regression techniques are discouraged, see Frank Harrell's commentary at http://www.stata.com/support/faqs/st...ems/index.html.

That said, there are a few things that can be done. Your code treats continuous_var1 and continuous_var2 as discrete variables, so you will get indicator variables for every separate value of each of those variables (save 1), and then these will be crossed with all levels of epoch, block, and condition! To have them treated as continuous variables, you need to put c. in front of them. That will already trim your model down considerably.

The next question is whether you really need to include all of the interactions among epoch, block and condition. Do you expect the effect of condition on your outcome to differ depending on which epoch and which block it was applied in? If so, then you do need those interaction terms (and you would still need them even if -stepwise- or some other "automagical" procedure for model simplification told you otherwise). But in many study designs, it is at least the intent and expectation that the effect of the condition will be the same across these (or perhaps not across block but across epoch, or the other way around) , so elimination of those interactions would make sense. Now, of course, I have no information about what your epochs, blocks, or conditions are, and even if I did I might not understand how they affect each other, so I can't advise you on which situation applies to you. But I would urge you to think carefully about these aspects of your study. To the extent that the effects of epoch, block, and condition might be expected to not depend on the others, you have the ability to remove those variables from the interaction and enter them just as main effects. Even removing just one of these from the interaction will considerably simplify your model.

Even if no simplification of the epoch/block/condition variables can be made, I also wonder if you really need for continuous variables 1 and 2 to be interacted with each other. Again, it is possible, but generally people try to design experiments to minimize dependencies between different aspects and it may be that there is no reason to expect that the effect of continuous variable1 depends on the value of continuous variable2 and vice versa. It may be that you only need them to be interacted with (some or all of) epoch/block/condition, and not with each other.

I really think your best bet is to ponder these issues and then choose your model accordingly. Model selection is best done with human judgment based on thoughtful understanding of the science and the circumstances.

[Andrew Teer]

Thank you for the thoughtful reply. I will definitely take all of this into consideration.