Can I use orthogonal transformation to fix multicollinearity in FEM?

[RASTI WIJAYANTI]

hello everyone, please help me
I am working with a panel dataset consisting of 219 observations (73 firms over 3 years). My model uses 5 independent variables (X1–X5) and 1 dependent variable (Y). I estimated the model using fixed effects (FEM).

When I checked multicollinearity using vif, I found that X1 and X5 have VIF values above 10, while the other variables are below the usual thresholds.

To address this, I orthogonalized X1 and X5 (to reduce their correlation with the other predictors). After orthogonalization, VIF values dropped to acceptable levels.

My questions:

1. Is orthogonalizing variables (like X1 and X5) acceptable practice in panel data models estimated via fixed effects?
2. After orthogonalization, should I still worry about multicollinearity when interpreting coefficients?
3. Is there a better approach to handle multicollinearity in panel data, besides orthogonalization (e.g., centering, dropping variables, or using ridge regression)?
4. Does the short time dimension (T=3) affect how VIF behaves in panel settings?

Any guidance on best practices in Stata for dealing with multicollinearity in panel FEM models would be very helpful.

Thank you.

[Manh Hoang Ba]

Hi RASTI,
As far as I know, the vif command doesn't run after FEM (xtreg, fe). I wonder if there's some misunderstanding?
Also, if the VIF is high but the coefficients are still statistically significant, then multicollinearity is probably not a major concern.

[Joseph Coveney]

As far as I know, the vif command doesn't run after FEM (xtreg, fe). I wonder if there's some misunderstanding?

I was wondering about that, too. I had to assume that Rasti was using regress with indicator variables for the 73 firms and that the predictors with the elevated variance-inflation factor were time-invariant (at the firm-level). And then I began to wonder just how does orthogonalization work with a mix of time-invariant and time-varying predictors? (Although nothing is mentioned in the post, I was guessing Rasti used the user-written orthog available on SSC.)

But my bigger question is along the lines of Hoang Ba's second comment, that is, does the regression model fitted after orthogonalization of the predictors give rise to conclusions that substantively differ from those of the model fitted using the original predictors? That, too, isn't mentioned in post.

[Manh Hoang Ba]

Hi Joseph Coveney,

I also guessed that orthogonalizing the variables using the orthog command would be helpful. But according to STATA documentation and accompanying references, this method's main purpose is to avoid computational problems when there is high multicollinearity. It doesn't actually help to fix multicollinearity.

More importantly, the initial parameter estimates (beta) are linear combinations of orthogonal parameters (gamma):
beta = R^-1*gamma'
Where X=Q*R, with Q being the vector of orthogonal variables and R being the triangular matrix.

Thus, gamma cannot be used to explain the meaning as a substitute for beta. Specifically, for example:
If gamma1 is statistically significant, it does not necessarily mean beta1 is statistically significant; and we also have no explanation: if X1 increases by 1 unit, then Y increases by gamma1 unit.

[Joseph Coveney]

I also guessed that orthogonalizing the variables using the orthog command would be helpful.

Thank you for reminding me that orthog has long since been made part of official Stata. I probably knew that at some point in time, but had forgotten.

. . . gamma cannot be used to explain the meaning as a substitute for beta. Specifically, for example:
If gamma1 is statistically significant, it does not necessarily mean beta1 is statistically significant; and we also have no explanation: if X1 increases by 1 unit, then Y increases by gamma1 unit.

But you don't lose anything: you can completely recover the original beta coefficients and their standard errors from the decomposition matrix provided by orthog, and so if you need to have an explanation or whatever in terms of the beta coefficients, then you are able to do so. I show how in the code below.

Begin at the "Begin here" comment; the top part just (i) creates an artificial dataset for illustration with three highly correlated predictors, (ii) shows the original fitted regression model, confirms the high values of the variance inflation factors (VIFs) and shows tests of its beta coefficients, and (iii) shows the regression model fitted to the orthogonalized predictors and that the VIFs are now unity. The code below the comment shows how to recover the original regression coefficients and their variance-covariance matrix, reconstructs the regression output table and confirms that the reconstructed coefficients and their standard errors (and Wald test statistics and confidence bounds) are the same as those shown in the original fitted model.

version 19

clear *

// seedem
set seed 1088277100

quietly drawnorm x1 x2 x3, double means(0 5 10) sd(1 2 4) ///
    corr(1 0.95 0.95 \ 0.95 1 0.95 \ 0.95 0.95 1) n(50)

generate double y = rnormal(10 * x1 + 5 * x2 - x3 + 100, 10)

regress y c.(x?)
estat vif
test x1
test x2
test x3

orthog x?, generate(o1 o2 o3) matrix(R)
regress y c.(o?)
estat vif

*
* Begin here
*
matrix define Ri = inv(R)
matrix define B = e(b) * Ri'
matrix define V = Ri * e(V) * Ri'
ereturn post B V, obs(`e(N)') dof(`e(df_r)')

ereturn display
test x1
test x2
test x3

exit

Do-file and corresponding log file are attached.

I think, though, that if OP is using orthog because of concern about "VIF values above 10" in anything under the rubric of General Linear Model that is fitted with modern software, then it's wasted effort.

[Clyde Schechter]

I think Joseph Coveney has done an outstanding job of responding to this thread, as he always does!

I just want to point out a different aspect of the situation. Let's call the orthogonally transformed variables of X1-X5 O1-O5. The variables X1-X5, I presume, have some meaning in the real world, which meaning motivated their inclusion in the model in the first place. Now, the variables O1-O5 are each linear combinations of X1-X5. These new variables therefore are each a mixture of the information drawn from X1-X5. In particular, these variables, individually, probably have no real-world interpretation at all. Could you assign meaningful names to these variables? Probably not? Maybe to one or two of them, in the right circumstances, but usually not. O1-O5 are purely mathematical constructs. Collectively they carry all of the information of X1-X5, but individually, they are usually opaque.

Now we can differentiate two cases. One case is where X1-X5 are introduced into the modeling solely to adjust for possible confounding effects or to reduce outcome variance. In that case, you are not trying to explain the outcome in terms of X1-X5, you are just trying to reduce the nuisance effects of X1-X5 on the outcome variable, and O1-O5 will do that to exactly the same extent. But, as you can see, you have not otherwise changed anything in the model: all you have accomplished is to beautify the VIF output, which, really, there is no good reason to care about.

The other case is that one or more of the X variables is actually a focal variable of interest in your situation. You are seeking to actively understand the relationship of that (those) variable(s) to the outcome. Now you are in deep trouble if you use O1-O5, because the information conveyed by the particular X('s) you are interested in is now scattered across the O1-O5 variables and can only be extracted by the kind of reverse transformation that you have been shown. In short, the use of the O1-O5 variables obscures the findings that are germane to the goal of your research. To achieve that goal, you must undo the transformation to recover the predictor/explanatory variable(s) of interest.

In short, while orthogonalization is a mathematical trick that can sometimes rescue problems with unstable or non-convergent estimations, it offers no benefit at all if the direct regression on X1-X5 produces usable results. It is, at best, unhelpful, and can even obscure the key findings of the study.

[RASTI WIJAYANTI]

I think Joseph Coveney has done an outstanding job of responding to this thread, as he always does!

I just want to point out a different aspect of the situation. Let's call the orthogonally transformed variables of X1-X5 O1-O5. The variables X1-X5, I presume, have some meaning in the real world, which meaning motivated their inclusion in the model in the first place. Now, the variables O1-O5 are each linear combinations of X1-X5. These new variables therefore are each a mixture of the information drawn from X1-X5. In particular, these variables, individually, probably have no real-world interpretation at all. Could you assign meaningful names to these variables? Probably not? Maybe to one or two of them, in the right circumstances, but usually not. O1-O5 are purely mathematical constructs. Collectively they carry all of the information of X1-X5, but individually, they are usually opaque.

Now we can differentiate two cases. One case is where X1-X5 are introduced into the modeling solely to adjust for possible confounding effects or to reduce outcome variance. In that case, you are not trying to explain the outcome in terms of X1-X5, you are just trying to reduce the nuisance effects of X1-X5 on the outcome variable, and O1-O5 will do that to exactly the same extent. But, as you can see, you have not otherwise changed anything in the model: all you have accomplished is to beautify the VIF output, which, really, there is no good reason to care about.

The other case is that one or more of the X variables is actually a focal variable of interest in your situation. You are seeking to actively understand the relationship of that (those) variable(s) to the outcome. Now you are in deep trouble if you use O1-O5, because the information conveyed by the particular X('s) you are interested in is now scattered across the O1-O5 variables and can only be extracted by the kind of reverse transformation that you have been shown. In short, the use of the O1-O5 variables obscures the findings that are germane to the goal of your research. To achieve that goal, you must undo the transformation to recover the predictor/explanatory variable(s) of interest.

In short, while orthogonalization is a mathematical trick that can sometimes rescue problems with unstable or non-convergent estimations, it offers no benefit at all if the direct regression on X1-X5 produces usable results. It is, at best, unhelpful, and can even obscure the key findings of the study.

Hello mr, thank you in advance for the discussion and helpful answers.

I am working with panel data and my main estimation uses a fixed effects model (xtreg, fe).
However, I am a bit confused about which Stata command should properly be used to diagnose multicollinearity in this context.

Since vif is not directly available after xtreg, fe, I see different practices in the literature, such as:
1. Running a pooled OLS model using: reg y x1 x2 x3 x4 vif
2. Running a fixed effects model and then using:xtreg y x1 x2 x3 x4, fe vif, uncentered

My question is:
Which command is methodologically more appropriate for checking multicollinearity when the final model is a fixed effects panel regression?

Should multicollinearity be assessed using pooled OLS (reg + vif) as a diagnostic tool, or is vif, uncentered after xtreg, fe acceptable and interpretable?

Any clarification or references would be greatly appreciated.

Thank you very much.

[Manh Hoang Ba]

Hello mr, thank you in advance for the discussion and helpful answers.

I am working with panel data and my main estimation uses a fixed effects model (xtreg, fe).
However, I am a bit confused about which Stata command should properly be used to diagnose multicollinearity in this context.

Since vif is not directly available after xtreg, fe, I see different practices in the literature, such as:
1. Running a pooled OLS model using: reg y x1 x2 x3 x4 vif
2. Running a fixed effects model and then using:xtreg y x1 x2 x3 x4, fe vif, uncentered

My question is:
Which command is methodologically more appropriate for checking multicollinearity when the final model is a fixed effects panel regression?

Should multicollinearity be assessed using pooled OLS (reg + vif) as a diagnostic tool, or is vif, uncentered after xtreg, fe acceptable and interpretable?

Any clarification or references would be greatly appreciated.

Thank you very much.

Run the vif, uncentered command after xtreg, fe doesn't provide the VIF of the FEM; instead, it returns the VIF coefficients of the Pooled OLS with the original variables, without constants.
To calculate the VIF of the FEM, we need to run regressions on time-demeaned data without constants.
The code below illustrates this:

webuse abdata, clear
* vif, uncentered not provide FE's VIF
qui xtreg n k w ys, fe
vif, uncentered
qui reg n k w ys, nocons
vif, uncentered
* compute FE's VIF
preserve
qui foreach var of varlist n k w ys {
    egen double `var'_m = mean(`var'), by(id)
    replace `var' = `var' - `var'_m // time-demeaned vars
}
qui reg n k w ys, nocons    // FE regression
vif, uncentered
restore

Results:

. webuse abdata, clear

. * vif, uncentered not provide FE's VIF
. qui xtreg n k w ys, fe

. vif, uncentered

    Variable |       VIF       1/VIF  
-------------+----------------------
          ys |    138.11    0.007241
           w |    137.72    0.007261
           k |      1.09    0.920391
-------------+----------------------
    Mean VIF |     92.30

. qui reg n k w ys, nocons

. vif, uncentered

    Variable |       VIF       1/VIF  
-------------+----------------------
          ys |    138.11    0.007241
           w |    137.72    0.007261
           k |      1.09    0.920391
-------------+----------------------
    Mean VIF |     92.30

. * compute FE's VIF
. preserve

. qui foreach var of varlist n k w ys {

. qui reg n k w ys, nocons        // FE regression

. vif, uncentered

    Variable |       VIF       1/VIF  
-------------+----------------------
           k |      1.27    0.788253
          ys |      1.27    0.788271
           w |      1.05    0.954505
-------------+----------------------
    Mean VIF |      1.19

. restore