regression with factors

[Alison Franck]

My cross-sectional regression analysis comes after reducing 3 sets of questions from a questionnaire to a smaller number of factors: 1 question with 11 alternatives now has 4 factors (all dependent variables of the regression), another question with 12 alternatives now has 5 factors (independent variables) and another question with 9 alternatives now has 2 factors (more independent variables). There are 4 regressions, in which the same independent variables are used in the 4 models, what changes is the dependent variable, the first one that went through factorial analysis and has 4 factors. In addition to the factors, there are 3 more categorical variables. All factors are continuous variables and the other variables are ordinal categorical variables!
Note: Before the factor analysis, all variables that went through factorization were positive and categorical, and then they became continuous with negative and positive values!

The functional form is more or less this one:

DV (= factor 1 on labor market) = b0 + independent variables (= factor2 .1 + factor2.2 + factor 2.3 + factor 2.4 + factor 2.5 + factor 3.1 + factor 3.2 + ordinal categorical variable 1 + ordinal categorical variable 2 + ordinal categorical variable 3)

DV (= factor 2 on labor market) = B0 + independent variables (= factor2 .1 + factor2.2 + factor 2.3 + factor 2.4 + factor 2.5 + factor 3.1 + factor 3.2 + ordinal categorical variable 1 + ordinal categorical variable 2 + ordinal categorical variable 3)

And so on for the other two regressions...
My problem is: I can't find a functional form that makes a valid regression, all tests are failing! Neither robust regression, nor quantile, nor GLM model. The data for all variables are non-normal, the regression has non-normal residuals, I can't use log for negative data, I really can't figure out how to analyze it! Has anyone worked with data like this and could give me a tip?

[Clyde Schechter]

The data for all variables are non-normal,...

This doesn't matter at all. You shouldn't even waste a second thinking about this, let alone fretting over it. The widespread belief that regression variables need to have normal distributions is a pernicious myth that results from poor teaching of statistics.

...the regression has non-normal residuals...

This sometimes matters, but only occasionally. It matters if you are using a linear regression and have a small sample size. But if your sample is even moderately large, the normality of residuals assumption is not needed to support inference. If your sample size is small enough that normality of residuals matters, then your sample size is also too small in other terms: it will seriously overfit a model with as many variables as you seek to use. If this is your situation, the solution is to get more data. If this is not your situation, then, again, you should just ignore this non-issue.

...I can't use log for negative data...

True. But in what you have described of your problem, even if the data were all positive, you don't say anything that suggests log-transforming anything would be helpful here.

all tests are failing!

Of all the "requirements" for a linear regression analysis, the only one that really matters in all situations is linearity--something that is rarely tested for, but always should be. How does your residuals vs fitted plot look? Is it consistent with an underlying linear relationship? Or does it suggest some non-linear relationship?

[Alison Franck]

"How does your residuals vs fitted plot look?" This is what they look like after I used "rreg " (robust regression), "predict yhat, xb", "predict resid, residuals" and "scatter resid yhat, yline(0) title("Residuals vs Fitted Values")". The residuals seem to be distributed quite concentrated around the zero line and range from -6 to 2. You'd expect them to be around 0, but I don't think they're in the right shape!

[Clyde Schechter]

I have only a little knowledge of the workings of -rreg-. I have the impression, though I may be wrong, that this procedure is rather deprecated these days. In any case, since part of what it does is remove outliers, a plot of residuals following it cannot give a valid impression of the fit of a linear regression to the data. Moreover, in principle, any time you are relying on the removal of outliers from your data (unless you have good evidence that the outliers are actually data errors) the modeling done on the remaining data is pretty much a fake. I know that's a strong, harsh way of putting it. But, look: you can only identify an outlier based on its outcome variable's deviation from the center of the data. So, given a new set of data, you cannot know which subset of the data will be retained and which will be discarded until you run it. So this kind of model cannot be used for prediction at all. Moreover as a model in support of understanding underlying relationships and mechanisms, it is similarly not useful because you cannot prospectively state to what subpopulation it can actually be applied. As this is not a model I have much experience with, it may be that I am overstating these problems or overlooking adequate workarounds for them. But I don't think I am, and I can't encourage anybody to use it. If others following along disagree, it would be good for them to speak up so we can all learn something more.

Meanwhile, I would recommend you calculate an ordinary linear regression with -regress- and then run -rvfplot- immediately afterward to see a residuals vs fitted plot that applies to all of the estimation sample.