back transformation of estimates after orthogonalization of variables

[Miguel Gonzalez-Loureiro]

Dear statalisters: hi to all.

My question is about the back transformation of estimates (coefficients, s.e. and p-values) after a regression that used the orthogonal transformation of variables with the orthog command.

In the statalist there are only 3-4 questions on orthogonalization, but none of them have dealt with this.

I used orthog feature of STATA to transform the variables in the regression (say oXs) and used the , matrix(R) command and matsave/matload to store /reload the matrix used for transformation. I did this because of high correlation and I needed to include interactions that created additional VIF problems. So I transformed all the Xs that were not dummies.

orthog X1 X2 X3 X4 X5   , gen (  oX1 oX2 oX3 oX4 oX5 )   matrix(R)

 matsave R, saving
 matload R, saving over

My goal is to present the table with the parameters of the original variables (variables 1 to 13 in the regression), which I did by following the help file of STATA (I was interested only on the estimates of transformed variables, thus the columns selected, plus the addition of the intercept estimated):

matrix B=e(b)[1,1..13]
matrix cte=e(b)[1,20]
matrix Bprim=(B, cte)
matrix b = Bprim*inv(R)'
matrix list b

This b matrix contains the back transformed estimates of coefficients in the original scale of each variable, which I will report in the paper.

My doubts are related to s.e. of coefficients and its t-statistics or p-values:

1. Some but not all the Xs are orthogonalized (for instance, I have dummy variables that don't need orthogonalization). Since the orthogonalization includes a constant, does it mean that the intercept of the orthogonalized regression should also be back transformed with the matrix? If you look at my code I really did it, since I saw the example in the help file and it includes the intercept, which is different to the orthogonalized estimate. Should the intercept be backtransformed when there is some orthogonalized variables in the regression (but not all of them, dummy variables are not transformed)?

2. I was able to present the coefficients back transformed. My question is whether standard errors of orthogonalized variables are the same for the back-transformed variables or whether I should apply the same code to obtain the standard error in the original scale of X variables. I noticed that the variance of a original X and the orthogonalized X are not the same (orthogonal variables have of course mean=0 and variance=1 or very close to it). Since s.e. of an estimate is equal to [ SSR/(n-2) * sqr(1/Var(X') ] and the Var(X') is not equal to Var(X), should I apply the same algebra to back transform s.e. ? If not, how could I compute that?

3. Should I compute p-values of the back transformed estimates or will they be the same as the orthogonal variables regression?. I mean, depending on the response to (2), I could compute the p-value of the back-transformed estimate (that should be the same as the orthogonalized variables regression). I have s.e. and the intercept, so I can compute the t-value and its p-value. But I'm unsure of whether this is needed in case the p-value be exactly the same.


Sorry if my question or code do not follow the FAQs, but it's my first time and did not know how to include that.

Thanks in advance

[Miguel Gonzalez-Loureiro]

Please, any help with the algebra involved in this? I would need to compute the s.e. and p-values of the back transformed estimates.
xtpcse uses Z stats for p-values, and not t
The help file of STATA does not provide guidance on this, but on the back transformation of coefficients, as posted above.
Thanks!

[Clyde Schechter]

I am one of the most algebra-oriented members of the Forum. I suppose I could figure this out, but it would take far more time than I have available for these purposes.

What I don't understand is why you don't just re-run the regression with the original un-orthogonalized variables: that will give you the "back-transformed" coefficients and standard errors with no real work at all.

Finally, have you read the chapter in Arthur Goldberger's A Course in Econometrics about multicolinearity. He makes the very useful point that most of the time "VIF problems" are not worth worrying about, and when they are, they are not solvable anyway, except by getting a different data set. You are probably putting yourself through a lot of hoops in vain with this entire exercise.

[Miguel Gonzalez-Loureiro]

Thanks for the orientation. I have read a similar comment from you, and I really found my VIFs were under acceptable thresholds. I just wanted to perform a robustness check by orthogonalizing the variables, perform the estimation and then backtransforming the estimates to compare that with the original estimation with unstranformed variables.

My "original" idea was to perform a backtransformation of standard error of estimates because xtpcse corrects standard errors for panel autocorrelation and heteroskedasticity.

So I assume that the s.e. of estimates of original variables will be different to those yielded by the estimates of ortoghonalized variables, both yileded by xtpcse. For the purpose of testing significance, the original estimates of unstranformed variables (coefficients) can be different in size to those yielded by the backtransformation and all the estimatations of s.e. and z-values are different. Am I right?.

If I understood you well, I could report the backtransformed estimates along with s.e. and z-val yielded by the same regression but with original variables (?). Did I understood you well? Or, are you suggesting to answer the reviewer with the VIF argumentation and forget totally the orthogonalization of variables? In the original version of the paper, we reported VIFs well below 5, but one reviewer commented we might have problems of collinearity (an issue I was confident we did not), so I was trying to show the reviewer that estimates were pretty similar with and without orthogonalization.

However, to my surprise, size of estimates and even in some case the sign is different (comparing original estimates of untransformed variables with backtransformed estimates). This is what intrigues me, and I want to be sure which approach is let's say "biased" or free from any type of misspecification. We performed all the tests (whether data is poolable, RE vs FE, heterosk, serial corr, contemporanous cross sectional corr and the like), and all of these tests led us to use xtpcse, with some lagged time-variant variables and some time-invariant ones.

To summarize: Clyde, would you suggest to totally forget the issue of orthogonalization and use the direct estimates of xtpcse of original variables? To me, if some with your expertise suggests this, it will give me confidence to follow that approach.

Thank you very much for the assistance and comment, because I was thinking on the way to use matrixes instead of reporting the direct s.e.

[Clyde Schechter]

For the purpose of testing significance, the original estimates of unstranformed variables (coefficients) can be different in size to those yielded by the backtransformation and all the estimatations of s.e. and z-values are different. Am I right?.

Yes, of course, that's right. It's a model with different variables and while the orthogonalized variables jointly carry exactly the same information as the original variables, their individual contributions to the regression will be different from those of the correspondingly numbered original variables. In particular, the information has been re-distributed so as to eliminate dependence among the variables. The relationships between the coefficients of the original and orthogonalized variables are not naively predictable: changes can be large or small and in any direction.

I could report the backtransformed estimates along with s.e. and z-val yielded by the same regression but with original variables (?). Did I understood you well? Or, are you suggesting to answer the reviewer with the VIF argumentation and forget totally the orthogonalization of variables? In the original version of the paper, we reported VIFs well below 5, but one reviewer commented we might have problems of collinearity (an issue I was confident we did not), so I was trying to show the reviewer that estimates were pretty similar with and without orthogonalization.

In #1 you did not mention that the regression was -xtpcse-. I don't know much about that command. If the panel corrected covariance matrix is not a linear transformation of the ordinary covariance matrix, then it may not be the case that back-transformed standard errors would be the same as the standard errors from the untransformed variables. I just don't know. But regardless, I do think that the reviewer's concern makes no sense. While I think multicollinearity is a grossly over-rated concept that is, in fact, ignorable in the most common situations, and I normally disparage the use of VIF, the fact that the VIFs were all so low is certainly adequate proof that, whatever you think of multicollinearity, it does not exist to any appreciable degree in your data. Perhaps the reviewer has information about your data and analyses that has not surfaced in this thread. But it is hard for me to imagine what that might be. In any case, if the reviewer is insistent that you prove to him/her that multicollinearity is not an issue for you and is unmoved by the standard errors of your key variables being sufficiently small (the real criterion for a multicollinearity problem), assuming they are, then showing the untransformed analysis and the orthogonalized analysis and finding that they lead to the same conclusions (even if some of the coefficients of the orthogonalized variables have changed considerably) should suffice. Either the back-transformed analysis would be the same as the untransformed one, or, if it isn't, I don't know what interpretation could be put on it that would be relevant to this issue.

[Miguel Gonzalez-Loureiro]

Great, thanks. VIFs were on average 1.79; the larger was 2.23. The largest correlation was 0.66, and some correlations are around 0.40. My guess is that some people tend to read correlations and do not consider the additional information of VIFs. Lasgest standard errors of unstranformed variables are as high as 0.2 or 0.5 when I perfom the regression in subsamples (by size, by age, just as a check) and have also three dummies with the largest s.e. (between 0.38 and 0.66). All in all, I know that some variables are not relevant, but we leave them simply to control for potential omitted variables bias, so although not significant, they capture a portion of explanation and make the other estimates more accurate (lower s.e.).

I think I have arguments to defend the position.

Sorry for not mentioning xtpcse. Really, the method corrects s.e. by specific panel AR1, heteros and cross panel correlation in the specification we use, since we detected those problems. I also eliminated singletons.

I know that back-transforming s.e. from this orthogonalization with xtpcse will require extra effort, if possible, because of that correction and the fact that orthog. IVs are combinations of original IVs. The average of each IV and observed variables will be affected by the transformation, whereas the observed DV is not. Prediction of DV can done and I can compute the squarred errors , but I cannot see how the matrix transforming the original IVs affects the computation of denominator for s.e. But yes, I cannot think of a clear utility of orthogonalizing variables, obtaining estimations and the s.e. & p-val, BUT one cannot use those estimates simply because those orthogonal variables are anything but the original ones. Thus, the estimate of a certain ortIV is not what many people may think, it is actually a certain combination of several other variables, so uninterpretable.

I would suggest other readers here to get rid of that and try other transformations than can yield variables for which estimators have meaning and inference on significance can be done. Otherwise, backtransforming coefficient estimates is very nice, but useless since no inference on significance can be done.

Thanks for your explanations and we can close this issue.