How to compare two correlation coefficients within the same sample?

[Stephen Bowman]

Let's say we have y being predicted by x1 and x2 such that y=a+b1*x1+b2*x2+e.
How do we compare b1 and b2 and if the differences between b1 and b2 are significant?

[Clyde Schechter]

Well, it is easy enough to test whether the difference between two coefficients in a model is "significant:"

regress y x1 x2
test x1 = x2

You can also look at the magnitude of the difference, along with a confidence interval using -lincom x1-x2-, which probably is more insightful and less prone to lead to paradoxes and fuzzy thinking.

The real question is: under what circumstances is such a comparison meaningful. Remember that if r represents the Pearson correlation between y and x, then in the regression model y = a + bx, b = r*sigma_y/sigma_x, where sigma_* are the standard deviations of y and x in the estimation sample, respectively. It's a little more complicated when you have more variables, but the same general principle applies: regression coefficients are dependent on the scale of variation (standard deviation) of the predictor variables. So in order for the comparison of two coefficients to make sense, the variables must be measured in the same units and must exhibit a similar scale of variation in the data. Otherwise, its apples to oranges.

An additional complication sets in if the variables x1 and x2 are themselves substantially correlated. When this is the case, the estimates of both b1 and b2 may be quite imprecise. While that does not preclude a sharp estimate of the difference b1-b2, it might. This is another situation where looking at the difference and its standard error or confidence interval gives much better insight than a significance test of whether b1 = b2.

[Stephen Bowman]

Thank you! We are trying to examine the convergent and discriminant validities of the x1 and x2 scales by linking them to y. The scales of x1 andx2 are measured in the same manner. In our case x1 and x2 are not correlated much luckily. But what should we do if they happen to be highly correlated?

[Phil Bromiley]

You may want to think a bit about your specification if you're trying to look at validity. The parameters on x1 and x2 in a regression with both x1 and x2 are based on the unshared variation in x1 and x2, but a lot of the validity issues center on shared variability. Many discussions of convergent and discriminant emphasis the association of the indicators rather than the indicators' association with another observed variable (which sounds a lot like nomological validity).