Correlations between predicted scores after orthogonally rotated PCA

[lloewens]

I am doing a pca on a correlation matrix. I predict the scores. Those scores are uncorrelated. I then conduct an orthogonal rotation and predict the scores again. The resulting scores are highly correlated. Shouldn't the orthogonally rotated scores be uncorrelated? I thought only an oblique rotation should result in correlations among the predicted scores?

Here is a minimum working example:


. sysuse auto, clear
(1978 Automobile Data)

. pca price mpg headroom length, components(2)

Principal components/correlation Number of obs = 74
Number of comp. = 2
Trace = 4
Rotation: (unrotated = principal) Rho = 0.8311

--------------------------------------------------------------------------
Component | Eigenvalue Difference Proportion Cumulative
-------------+------------------------------------------------------------
Comp1 | 2.432 1.53951 0.6080 0.6080
Comp2 | .89249 .411322 0.2231 0.8311
Comp3 | .481168 .286827 0.1203 0.9514
Comp4 | .194341 . 0.0486 1.0000
--------------------------------------------------------------------------

Principal components (eigenvectors)

------------------------------------------------
Variable | Comp1 Comp2 | Unexplained
-------------+--------------------+-------------
price | 0.3977 -0.7152 | .1589
mpg | -0.5745 0.0703 | .193
headroom | 0.4094 0.6918 | .1654
length | 0.5867 0.0709 | .1582
------------------------------------------------

. predict pc1 pc2, score

Scoring coefficients
sum of squares(column-loading) = 1

----------------------------------
Variable | Comp1 Comp2
-------------+--------------------
price | 0.3977 -0.7152
mpg | -0.5745 0.0703
headroom | 0.4094 0.6918
length | 0.5867 0.0709
----------------------------------

. pwcorr pc1 pc2, sig

| pc1 pc2
-------------+------------------
pc1 | 1.0000
|
|
pc2 | 0.0000 1.0000
| 1.0000
|

. rotate

Principal components/correlation Number of obs = 74
Number of comp. = 2
Trace = 4
Rotation: orthogonal varimax (Kaiser off) Rho = 0.8311

--------------------------------------------------------------------------
Component | Variance Difference Proportion Cumulative
-------------+------------------------------------------------------------
Comp1 | 1.75596 .187427 0.4390 0.4390
Comp2 | 1.56853 . 0.3921 0.8311
--------------------------------------------------------------------------

Rotated components

------------------------------------------------
Variable | Comp1 Comp2 | Unexplained
-------------+--------------------+-------------
price | -0.1761 0.7991 | .1589
mpg | -0.3836 -0.4333 | .193
headroom | 0.7650 -0.2468 | .1654
length | 0.4864 0.3357 | .1582
------------------------------------------------

Component rotation matrix

----------------------------------
| Comp1 Comp2
-------------+--------------------
Comp1 | 0.7489 0.6627
Comp2 | 0.6627 -0.7489
----------------------------------

. predict rpc1 rpc2, score

Scoring coefficients for orthogonal varimax rotation
sum of squares(column-loading) = 1

----------------------------------
Variable | Comp1 Comp2
-------------+--------------------
price | -0.1761 0.7991
mpg | -0.3836 -0.4333
headroom | 0.7650 -0.2468
length | 0.4864 0.3357
----------------------------------

. pwcorr rpc1 rpc2, sig

| rpc1 rpc2
-------------+------------------
rpc1 | 1.0000
|
|
rpc2 | 0.4604 1.0000
| 0.0000

[lloewens]

This does not seem to happen in R (see code below). Is something wrong with the Stata rotate command?

data(mtcars)
#Initial PCA pca <- prcomp(mtcars, rank = 2) summary(pca)
## Importance of first k=2 (out of 11) components: ## PC1 PC2 ## Standard deviation 136.533 38.14808 ## Proportion of Variance 0.927 0.07237 ## Cumulative Proportion 0.927 0.99937
pca$rotation
## PC1 PC2 ## mpg -0.038118199 0.009184847 ## cyl 0.012035150 -0.003372487 ## disp 0.899568146 0.435372320 ## hp 0.434784387 -0.899307303 ## drat -0.002660077 -0.003900205 ## wt 0.006239405 0.004861023 ## qsec -0.006671270 0.025011743 ## vs -0.002729474 0.002198425 ## am -0.001962644 -0.005793760 ## gear -0.002604768 -0.011272462 ## carb 0.005766010 -0.027779208
scores <- predict(pca, mtcars) cor(scores)
## PC1 PC2 ## PC1 1.000000e+00 -8.203564e-16 ## PC2 -8.203564e-16 1.000000e+00
#Rotation pca2 <- principal(mtcars, nfactors = 2, rotate = "varimax") summary(pca2)
## ## Factor analysis with Call: principal(r = mtcars, nfactors = 2, rotate = "varimax") ## ## Test of the hypothesis that 2 factors are sufficient. ## The degrees of freedom for the model is 34 and the objective function was 2.95 ## The number of observations was 32 with Chi Square = 74.21 with prob < 8.1e-05 ## ## The root mean square of the residuals (RMSA) is 0.05
scores2 <- predict(pca2, mtcars) cor(scores2)
## RC1 RC2 ## RC1 1.000000e+00 1.025542e-15 ## RC2 1.025542e-15 1.000000e+00

[Nick Cox]

If you rotate orthogonally back from the PCs to (linear scalings of) the original variables, then those are correlated. So I offer that as a counter-example.