How to get BETA coefficients for Hierarchical Linear Model?

[Minda Tan]

My model has three levels including personal level, class level, and school level. I used "xtmixed" command to run HLM. But, it only showed Bs. How can I get beta coefficients for my model?

I learned that I can get the standardized result through standardizing dependent and independent variables. But, due to I have three levels, I'm not sure how can I compute them...

[Clyde Schechter]

But, due to I have three levels, I'm not sure how can I compute them...

This is precisely the problem: it is never clear at what level the standardization should be calculated. And that is one important reason why standardized coefficients are rarely used in hierarchical linear models.

Even in one-level models, standardized coefficients are really seldom helpful, and more often than not, just obfuscatory.

Why do you want standardized coefficients? If you can explain how they will be helpful in fulfilling your research goals and communicating results in your context, perhaps a sensible way to do them can be found.

[Minda Tan]

This is precisely the problem: it is never clear at what level the standardization should be calculated. And that is one important reason why standardized coefficients are rarely used in hierarchical linear models.

Even in one-level models, standardized coefficients are really seldom helpful, and more often than not, just obfuscatory.

Why do you want standardized coefficients? If you can explain how they will be helpful in fulfilling your research goals and communicating results in your context, perhaps a sensible way to do them can be found.

Thanks for responding to my question. I'm trying to analyze multi-level factors' effect on students' academic performance. One of my committee member thought the size of the coefficient was not really informative. He wanted me to make them more comparable. That's why he suggested me to report beta instead of b.

[Clyde Schechter]

One of my committee member thought the size of the coefficient was not really informative. He wanted me to make them more comparable. That's why he suggested me to report beta instead of b.

Well, as I indicated in #2, that's unlikely to be true. But what are the variable concerned her? What is the predictor for which it is though that the coefficient would be more informative if standardized? And what is the outcome variable in the model? And how does that variable relate to the levels in the model? (Actually, it would also be helpful to say what the levels in the model are.)

[Minda Tan]

Well, as I indicated in #2, that's unlikely to be true. But what are the variable concerned her? What is the predictor for which it is though that the coefficient would be more informative if standardized? And what is the outcome variable in the model? And how does that variable relate to the levels in the model? (Actually, it would also be helpful to say what the levels in the model are.)

For example, the outcome variable is math test score of grade 9 students.The predictors include 1) personal level: parental income, parenting style, and left-behind children (dichotomous); 2) class level: class advisers' education level and profession level; 3) school level: school SES and public school (dichotomous). Among them, some variables are continuous like income and SES, but their scales are largely varied.

[Bruce Weaver]

Even in one-level models, standardized coefficients are really seldom helpful, and more often than not, just obfuscatory.

Why do you want standardized coefficients? If you can explain how they will be helpful in fulfilling your research goals and communicating results in your context, perhaps a sensible way to do them can be found.

I agree with Clyde Schechter. Here is a note I wrote for myself about some similar comments in John Fox's book on regression models.

--- Start of note ---

John Fox, author of "Applied Regression Analysis and Generalized Linear Models" (2008, Sage) is very cautious about the use of standardized regression coefficients. He gives this interesting example. When two variables are measured on the same scale (e.g., years of education, and years of employment), the relative impact of the two can be compared directly. But suppose those two variables differ substantially in the amount of spread. In that case, comparison of the standardized regression coefficients would likely yield a very different story than comparison of the raw regression coefficients. Fox (p. 95) then says:

If expressing coefficients relative to a measure of spread potentially distorts their comparison when two explanatory variables are commensurable [i.e., measured on the same scale], then why should the procedure magically allow us to compare coefficients [for variables] that are measured in different units?

That is an excellent question!

--- End of note ---

HTH.

[Clyde Schechter]

So, here are the general principles that I recommend.

1. The purpose of reporting regression results is to gain understanding of how strongly differences in a predictor variable are associated with differences in the outcome variable.

2. If a variable is already defined in units that are meaningful and widely understood, there is nothing to be gained by standardizing it. In fact, that will only make the results more confusing and it should be actively avoided!

2.1 As a corollary, categorical (including dichotomous) variables should never be standardized.

3. If a variable is inherently dimensionless because, for example, it represents a ratio of two variables that have the same units, or because it is the ratio of a result to an absolute (and widely recognized) standard value, then, again, standardizing the variable will add, not remove, confusion.

4. If a variable is measured in arbitrary units that are understandable only to those intimately familiar with the specific measurement, then for presentation to other audiences, clarity can be gained by recasting the variable in terms of some population based measurement, of which a standardized variable is among the simplest and most popular.

How do these apply here?

The dependent variable here is a math test score. So it depends on what the math test is. If it's a homebrew test made up for the study, then standardizing would be a very good idea. Without standardization, nobody could possibly know whether, say, a 5 point difference is a big deal or just a blip. Standardization would be one way to put the results in perspective. If, however, the math test is a standardized exam that has population norms, such as, for example, the SBAC tests used in California, then the variable is already given in meaningful population-based terms that most educators would understand and should be left alone.

Parental income would normally be measured in dollars (or whatever the local currency unit is) and would best be left alone; standardizing would only be confusing. Only somebody familiar with your data set would be able to interpret a 1 standard deviation change in income!

I have no sense of how you might have measured parenting style. If it is categorical, certainly leave it alone. If it is some scale, then my attitude towards it would depend on whether it is a widely used scale, whose scores would be readily understood by people in your field (leave it alone) or if it is some idiosyncratic measurement (definitely standardize it).

Class advisers' education level are presumably measured as years of education (or perhaps categories derived from that), and professional level is presumably categorical, so these should be left as they are.

School SES again depends on how it is measured. In school-based studies I have read, this is typically measured as the percentage of the school's students who are eligible for subsidized lunch. That unit is quite clear on its own terms, and standardizing it would just confuse things. But you may be using some other measure of school SES. If your measure of SES is one that is not widely understood in your discipline, then, once again, assuming it is continuous, standardize it. If it is categorical, or based on a widely known metric, then let it be. Again, public school is a dichotomy and should not be standardized.

Variability among the scales is never an argument for or against standardizing variables. Sometimes it makes sense to change the units of a variable to make the coefficient's significant digits fall close to the decimal point. For example, if your income is denominated in dollars and the coefficient comes out to be 0.000015, it might well make sense to rescale it in units of $10,000, so that the coefficient will change to 0.15, an easier number to grasp. But standardizing would only detract from the clarity of a straightforward variable like income.

As for how to standardize, I would standardize any variable at the level for which it is defined. So for a bottom level variable, I would standardize over the full sample of data. For a variable defined at, say, the school level, I would flag one observation per school, and standardize over that sample, so that each school contributes one and only one observation to the standardization calculations, and the standard deviation used does not reflect the automatic, and meaningless, absence of variation within school.

Your committee member may disagree with my advice, and ultimately you must answer to him or her, not to me. And only you know that committee member well enough to know whether he or she will be open to the arguments I have raised here. Even if ultimately you are pressed to depart from my advice, I hope it will be worth while having considered it, and perhaps you will follow it when you are working more independently in the future.

[Minda Tan]

I agree with Clyde Schechter. Here is a note I wrote for myself about some similar comments in John Fox's book on regression models.

--- Start of note ---

John Fox, author of "Applied Regression Analysis and Generalized Linear Models" (2008, Sage) is very cautious about the use of standardized regression coefficients. He gives this interesting example. When two variables are measured on the same scale (e.g., years of education, and years of employment), the relative impact of the two can be compared directly. But suppose those two variables differ substantially in the amount of spread. In that case, comparison of the standardized regression coefficients would likely yield a very different story than comparison of the raw regression coefficients. Fox (p. 95) then says:


That is an excellent question!

--- End of note ---

HTH.

Thank you so much!

[Minda Tan]

So, here are the general principles that I recommend.

1. The purpose of reporting regression results is to gain understanding of how strongly differences in a predictor variable are associated with differences in the outcome variable.

2. If a variable is already defined in units that are meaningful and widely understood, there is nothing to be gained by standardizing it. In fact, that will only make the results more confusing and it should be actively avoided!

2.1 As a corollary, categorical (including dichotomous) variables should never be standardized.

3. If a variable is inherently dimensionless because, for example, it represents a ratio of two variables that have the same units, or because it is the ratio of a result to an absolute (and widely recognized) standard value, then, again, standardizing the variable will add, not remove, confusion.

4. If a variable is measured in arbitrary units that are understandable only to those intimately familiar with the specific measurement, then for presentation to other audiences, clarity can be gained by recasting the variable in terms of some population based measurement, of which a standardized variable is among the simplest and most popular.

How do these apply here?

The dependent variable here is a math test score. So it depends on what the math test is. If it's a homebrew test made up for the study, then standardizing would be a very good idea. Without standardization, nobody could possibly know whether, say, a 5 point difference is a big deal or just a blip. Standardization would be one way to put the results in perspective. If, however, the math test is a standardized exam that has population norms, such as, for example, the SBAC tests used in California, then the variable is already given in meaningful population-based terms that most educators would understand and should be left alone.

Parental income would normally be measured in dollars (or whatever the local currency unit is) and would best be left alone; standardizing would only be confusing. Only somebody familiar with your data set would be able to interpret a 1 standard deviation change in income!

I have no sense of how you might have measured parenting style. If it is categorical, certainly leave it alone. If it is some scale, then my attitude towards it would depend on whether it is a widely used scale, whose scores would be readily understood by people in your field (leave it alone) or if it is some idiosyncratic measurement (definitely standardize it).

Class advisers' education level are presumably measured as years of education (or perhaps categories derived from that), and professional level is presumably categorical, so these should be left as they are.

School SES again depends on how it is measured. In school-based studies I have read, this is typically measured as the percentage of the school's students who are eligible for subsidized lunch. That unit is quite clear on its own terms, and standardizing it would just confuse things. But you may be using some other measure of school SES. If your measure of SES is one that is not widely understood in your discipline, then, once again, assuming it is continuous, standardize it. If it is categorical, or based on a widely known metric, then let it be. Again, public school is a dichotomy and should not be standardized.

Variability among the scales is never an argument for or against standardizing variables. Sometimes it makes sense to change the units of a variable to make the coefficient's significant digits fall close to the decimal point. For example, if your income is denominated in dollars and the coefficient comes out to be 0.000015, it might well make sense to rescale it in units of $10,000, so that the coefficient will change to 0.15, an easier number to grasp. But standardizing would only detract from the clarity of a straightforward variable like income.

As for how to standardize, I would standardize any variable at the level for which it is defined. So for a bottom level variable, I would standardize over the full sample of data. For a variable defined at, say, the school level, I would flag one observation per school, and standardize over that sample, so that each school contributes one and only one observation to the standardization calculations, and the standard deviation used does not reflect the automatic, and meaningless, absence of variation within school.

Your committee member may disagree with my advice, and ultimately you must answer to him or her, not to me. And only you know that committee member well enough to know whether he or she will be open to the arguments I have raised here. Even if ultimately you are pressed to depart from my advice, I hope it will be worth while having considered it, and perhaps you will follow it when you are working more independently in the future.

It's very helpful! Now I'm trying to standardize some of my variables according to your suggestion and I will discuss it with my adviser tomorrow. Thanks a lot! Really appreciate it!