I’m advising a colleague on an analysis for which the goal is to compare the factor structure of an approximately 20-item attitude scale administered to respondents in different countries. There are some known confounders (age, sex, etc.) whose distributions differ across countries, and we’re looking for a way to compare the factor structure of this scale while adjusting for these confounders. I’ve been trying a matching and a propensity scoring approach, neither of which has worked well, as I’ll describe below. I’d be interested in any suggestions or reactions people might have. For concreteness, I’ll describe the problem as one of comparing U.S. and Chinese respondents.
We first tried a matched analysis, with Chinese respondents individually matched to the U.S. respondents on the confounders. We then performed the analysis separately on the Chinese and U.S. parts of the matched sample, and compared the factor structure results (loadings, etc.). The matched sample size was ok, but failures to match were severe enough that the confounder distributions in each part of the matched sample as to make the comparison uninteresting. ("If Chinese and U.S. respondents counterfactually had a distribution of confounders that matched each other but were quite different from either one's original sample, would they have the same factor structure? )
We also tried a propensity scoring approach, with weighting. We generated a U.S =1 vs. China = 0 propensity score with the various confounders as predictors. We then did the factor analysis on the whole U.S. sample without weighting, and on the whole Chinese sample using the propensity score as a weight--i.e., "Would the Chinese factor structure look like the U.S. factor structureif the Chinese sample were made made to have a confounder distribution like the U.S. sample?” The difficulty here was that even the propensity-weighting did not balance the confounders, i.e, the weighted Chinese sample still did not have even univariate confounder distributions that came close to those for the original U.S. sample.
Any thoughts here? Maybe the methods are OK, but simply not workable in the current case? Maybe there is something else to try?
We first tried a matched analysis, with Chinese respondents individually matched to the U.S. respondents on the confounders. We then performed the analysis separately on the Chinese and U.S. parts of the matched sample, and compared the factor structure results (loadings, etc.). The matched sample size was ok, but failures to match were severe enough that the confounder distributions in each part of the matched sample as to make the comparison uninteresting. ("If Chinese and U.S. respondents counterfactually had a distribution of confounders that matched each other but were quite different from either one's original sample, would they have the same factor structure? )
We also tried a propensity scoring approach, with weighting. We generated a U.S =1 vs. China = 0 propensity score with the various confounders as predictors. We then did the factor analysis on the whole U.S. sample without weighting, and on the whole Chinese sample using the propensity score as a weight--i.e., "Would the Chinese factor structure look like the U.S. factor structureif the Chinese sample were made made to have a confounder distribution like the U.S. sample?” The difficulty here was that even the propensity-weighting did not balance the confounders, i.e, the weighted Chinese sample still did not have even univariate confounder distributions that came close to those for the original U.S. sample.
Any thoughts here? Maybe the methods are OK, but simply not workable in the current case? Maybe there is something else to try?
