Positive at spearman while negative at regression

[Paul Leeds]

Hello everyone,

This issue kept me all day wondering if it's make any sense.
In my spearman test I had the dependent and the main independent variables positively correlated while they are negatively related at the random effect regression results!

Does that make any sense?

[Paul Leeds]

More clarification.

X is the years of relationship (1-9 years) while Y is an ordinal variable which indicate the quality of annual reports (1-30) high denotes low quality.


In theory X should has a negative relationship with Y, which the regression results came in line with.
However, the Spearman correlation (positive correlation) is the issue of concern to me.

Hope this helps.

Thanks

[Clyde Schechter]

Since you are doing a random effects regression, I assume there is some longitudinal structure to your data with series of observations within units of analysis. You can have a situation where the within unit correlation is negative but the between-unit correlation is positive. A random-effects regression coefficient is a blend of the between- and within- unit effects, whereas the Spearman correlation is simply ignorant of the structure and could be almost anything! Try graphing your data with -xtline, overlay- and you may get a visual sense of what is happening.

[Paul Leeds]

Hello Clyde

Thanks for your reply.
If you mean graphing Y to see "within and between-unit behaviour" so this is it.
Does it reveal much?

I hope the graph is clear.

[Clyde Schechter]

Well, I guess that doesn't really show us much. Too many overlapping lines. It might be easier to see what's going on if you just did the graph on a random sample of a few dozen units.

Even with the graph as jumbled as it is, it doesn't like there's much of a positive or negative relationship here. Whatever is there, it's a small effect.

Another thought would be to pretend your Y variable is not just ordinal, and do -xtreg Y X, fe- to get the within-unit association, and -xtreg Y X, be- to get the between unit association and see if they are of opposite signs.

[ben earnhart]

Do you have additional variables in the random effects model, or are you presenting the baseline without additional variables? The moment you start throwing in additional variables, things can change.

[Carlo Lazzaro]

Paul:
as an aside to previous sound insights, under a regression framework the relationship between the dependent variable and the main predictor is adjusted for other predictors included in the right hand side of the equation. Hence, no wonder that the sign of the relationship may flip when contrasted against the results of Spearman test.

[Paul T Seed]

As well as the points mentioned, I would be interested in the distributions. What do you get if you carry out a Pearson's correlation. If the sign is different from the Spearman's correlation, that would imply you have a non-Normal distribution, and may have to transform.
Ir R and rho are roughly the same, the problem may line in the between-group and within-group issues mentioned by others.

BW, Paul

[Paul Leeds]

Well, I guess that doesn't really show us much. Too many overlapping lines. It might be easier to see what's going on if you just did the graph on a random sample of a few dozen units.

Even with the graph as jumbled as it is, it doesn't like there's much of a positive or negative relationship here. Whatever is there, it's a small effect.

Another thought would be to pretend your Y variable is not just ordinal, and do -xtreg Y X, fe- to get the within-unit association, and -xtreg Y X, be- to get the between unit association and see if they are of opposite signs.

Hi Clyde,

Yes, as per your speculation; the within unit is significantly positive, while the between unit is insignificant and negative.
Any thoughts?

Many Thanks

Paul

[Paul Leeds]

Do you have additional variables in the random effects model, or are you presenting the baseline without additional variables? The moment you start throwing in additional variables, things can change.

Yes of course. I noticed that the sign is only positive in the random effects model when including only the main effect but when adding the year dummy in particular the sign flips to positive!
Does that indicate the that the effect of the panel structure has its say on the sign?

Thanks

[Paul Leeds]

Paul:
as an aside to previous sound insights, under a regression framework the relationship between the dependent variable and the main predictor is adjusted for other predictors included in the right hand side of the equation. Hence, no wonder that the sign of the relationship may flip when contrasted against the results of Spearman test.

It is getting clearer now.

Thanks Carlo

[Paul Leeds]

As well as the points mentioned, I would be interested in the distributions. What do you get if you carry out a Pearson's correlation. If the sign is different from the Spearman's correlation, that would imply you have a non-Normal distribution, and may have to transform.
Ir R and rho are roughly the same, the problem may line in the between-group and within-group issues mentioned by others.

BW, Paul

Hello Paul,

I have done the pearson as well and it turn out to be the same sign (positive)
Does that rule out any possibility of non linearity?

Many Thanks

[ben earnhart]

Yes of course. I noticed that the sign is only positive in the random effects model when including only the main effect but when adding the year dummy in particular the sign flips to positive!
Does that indicate the that the effect of the panel structure has its say on the sign?

Yup. See for example, this excellent discussion by our own Richard Williams: http://www3.nd.edu/~rwilliam/stats2/l35.pdf . In brief (for one of his examples), the bivairate correlation between Head Start and later outcomes may be negative, since participants come from low SES households. But if you can account for SES, the program probably helps people, and will have a positive sign in a full regression model.

[Paul Leeds]

Thanks a lot ben.

Appreciate all the valuable comments.