combining likert scaled data into one variable

[Isabella BP]

Hi,

I have some survey data that I wish to analyze. The dependent variable is measuring a person's vulnerability and for this, I wish to combine 5 Likert scaled questions into one variable to form thisdependent variable. My question is a) can I do this and b) if so, shall I combine the variables using factor analysis or as a sum of the mean?

Thank you!

[Weiwen Ng]

Is this a validated scale, I.e. other researchers have published psychometric studies on it?

If not, the PI should be asked why a validated scale was not used. (If you are a research assistant, there may not be a good way to do this.) You can at minimum conduct an EFA to show that the scale looks unidimensional, and maybe show Cronbach’s Alpha (which doesn’t show unidimensionality, really, but people often ask for it). But in general, you don’t want to just cook up a bunch of questions. You want to first validate that the instrument measures what it purports to measure, and that it does it well.

if yes, then just sum the scores. This is done in just about every randomized trial in the medical field. This approach has theoretical flaws, but it appears to serve well enough. There are more advanced methods to do it, but many are a lot harder to learn. Factor scores should be acceptable, but most people aren’t used to interpreting them.

[Isabella BP]

Thank you, this is for the construction of a novel index which has not been done before so there are no other studies on it. So do you recommend I sum each of the scores (from 1-10) for each of the variables resulting in a final score which will become the dependent variable? So e.g. likelihood of installing new technology + likelihood of ... + xx + xx = dependent variable? Do I then need to rescale it to then perform a regression using the independent variables?

[Weiwen Ng]

Thank you, this is for the construction of a novel index which has not been done before so there are no other studies on it. So do you recommend I sum each of the scores (from 1-10) for each of the variables resulting in a final score which will become the dependent variable? So e.g. likelihood of installing new technology + likelihood of ... + xx + xx = dependent variable? Do I then need to rescale it to then perform a regression using the independent variables?

1. The PI needs to consider why they had to construct a novel instrument. Is there no suitable instrument that has been used before? If there is, why was that one not used?

2. I would strongly prefer to validate the existing instrument. That means EFA at minimum, plus discuss some theory for why the individual questions were chosen. I would prefer the researchers run a CFA and report the goodness of fit statistics. If I were reviewing a paper with a completely new instrument, I would definitely question things if the items seemed to be poorly selected or the construct you are trying to measure is ill-defined. For example, you say you want to measure a person's vulnerability. How is that broad construct defined? Is there a more clearly defined version of the construct? For example, in aging studies, the Linda Fried et al version of frailty index is relatively narrow and focuses on physical frailty. The Rockwood et al conceptual model of frailty is much broader and uses a lot more indicators. Either are OK, because the authors did background work discussing their theories, showing things like construct validity, and showing predictive validity (i.e. their instruments are correlated with higher mortality rates). So, for some measure of vulnerability, I would also like to see if it predicts things that you would expect to flow from being vulnerable, e.g. death, hospitalization, etc.

3. If you then use the vulnerability index, you can just sum the scores. The thing is that other parties need some way to interpret what one point really means. Hence, ideally, you want to have shown some sort of relationship between vulnerability and other outcomes, so you would have evidence that, e.g. a 10-point increase in vulnerability is associated with a hazaard rate of 1.3 for mortality (made up #s and outcomes). Otherwise you are just showing a scale and others have no real way to interpret what a 1-point change really means.

[Richard Williams]

Some basic approaches to scale construction are explained in

https://www3.nd.edu/~rwilliam/stats2/l23.pdf

However, do as much as you can of what Weiwen suggests.

[Isabella BP]

Thank you both

[Isabella BP]

There is no instrument that has been used before as we have created this survey ourselves and are in the process of running the survey now. The 5 questions asked are specific to this project and measure a certain type of vulnerability in being left behind in the energy system transition and include likelihood of installing new technologies, likelihood of using less energy, usage of battery and usage of smart meters.

I have taken a look at that link and done a factor analysis. Using the 5 variables, alpha looks okay. Just to double check - when you say "sum the scores" do you simply mean adding each of the variables and using the total of each of the scores? Or do you mean using alpha's scale variable?

Thanks

[Richard Williams]

If items are all coded the same way and forming a scale seems justified, then simply summing is often best. If they are coded differently you may need to standardize the items.