Factor Analysis: a methodological question

[Marie Sofie]

HI. Perhaps it is not the most adequate platform for my question, for which I beg you pardon, but I am a bit desperate. So, I have several batteries of questions in my survey data. I have run some factor analyses in Stata and calculated the Cronbach's alpha, etc. and have seen that in one of these measurements, two items out of five items do not load well in the factor (<0.40). Since these items have really low loadings, I would like to delete them from the factor and create an index only out of the items which load well in the factor. Is it ok to do that? Or would I need some more additional tests? I would really appreciate your help on that. Thanks a bunch.

[Clyde Schechter]

If these are batteries of questions that have been previously developed, tested, and are in general usage in populations similar to yours and for purposes similar to yours, I would be loathe to modify them on the basis of loadings in one sample. That could just be a fluke, say, sampling variation. Or perhaps it points to errors in your data in the variables that are expected to load well but don't.

But if these are "homebrew" batteries with no track record, or if you are using them in a situation or population that is materially different from the ones in which they were developed, then, it may be reasonable to omit items that load poorly if there is no strong theoretical basis for retaining them. The separate items excluded may still be useful on their own in your analyses and combining them with other variables that they don't really fit well with may just be throwing away information. Also, if these are homebrew batteries, have you looked at rotated solutions to your factor analysis? Sometimes items that don't load well in the initial solution, do fit in somewhere after rotations are applied. Also, have you looked at what happens to Cronbach's alpha when exclude the variables?

To make this long story short: if you are in the process of developing these batteries, then you have considerable latitude. But if you are using well-established instruments in domains and populations where they have a good track record, you need a strong justification to tamper with them.

Added: One more thought. Don't make a fetish out of the 0.4 loading threshold. That's no better than the widespread p < 0.05 fetish. The 0.4 threshold is a rule of thumb; it is not a law of nature. If your loadings are close to 0.4, the case for removing them is not compelling.

[Marie Sofie]

Dear Clyde! It is indeed a beautiful response! It is super clear to me now and I have indeed kept all the items, since the measurements are validated ones and have been used for many years. What I will do as well is to run the analysis with a shortened measurement as well, for robustness check, just to make sure it gives similar results. Thanks a lot indeed.