Can control variables change point estimates for margins, over(group)?

[Weston Ley]

I'm trying to calculate the marginal mean for each level of a categorical variable while controlling for confounding factors. I was under the impression that I could do this using the margins, over(group) command; however, introducing control variables has no effect on the marginal means.

This problem can be illustrated using a dataset released by UCLA:

use https://stats.idre.ucla.edu/stat/data/hsbdemo, clear
qui: reg female i.ses
margins, over(ses)
qui: reg female i.ses read write
margins, over(ses)
qui: reg female i.ses read write i.prog
margins, over(ses)

If you run these commands, you will notice that the point estimates for the marginal means do not change upon introducing control variables (although the standard errors do change). Why is this the case? By contrast, the command margins group does return different estimates when control variables are introduced. For example, the commands below return different point estimates:

use https://stats.idre.ucla.edu/stat/data/hsbdemo, clear
qui: reg female i.ses
margins ses
qui: reg female i.ses read write
margins ses
qui: reg female i.ses read write i.prog
margins ses

Should I be using the margins group command instead of margins, over(group)? If so, why?

[Clyde Schechter]

-margins- by itself will calculate the mean value of -regress-'s calculation of xb. That is, if you followed any of the regressions with

predict xb
tabstat xb, by(ses)

you would get the same point estimates you got from -margins, over(ses)-. That's what -margins, over()- does.

Now do that with each of the regressions and you will see that the mean predicted value of xb is the same regardless of what covariates you include in the regression. Why is that? Well, it is a standard fact about linear regressions that the mean value of xb is equal to the mean of y (the outcome variable, in your case female). And when you have a linear regression on a categorical variable (i.ses in your case) that is tantamount, at least as far as the calculation of xb is concerned, to doing a separate linear regression in each subset of the data defined by a value of that categorical variable. The fact that you are including read and write doesn't change the mean value of xb at all, even though it changes the individual observation-level values of xb.

If what you want is ses-conditional mean values, crude, adjusted for read and write, and for read, write, and i.prog, what you have to do is omit i.ses from the regression.

. use https://stats.idre.ucla.edu/stat/data/hsbdemo, clear
(highschool and beyond (200 cases))

. qui reg female

. margins, over(ses)

Predictive margins                                         Number of obs = 200
Model VCE: OLS

Expression: Linear prediction, predict()
Over:       ses

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
         ses |
        low  |       .545   .0353002    15.44   0.000     .4753895    .6146105
     middle  |       .545   .0353002    15.44   0.000     .4753895    .6146105
       high  |       .545   .0353002    15.44   0.000     .4753895    .6146105
------------------------------------------------------------------------------

. qui reg female  read write

. margins, over(ses)

Predictive margins                                         Number of obs = 200
Model VCE: OLS

Expression: Linear prediction, predict()
Over:       ses

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
         ses |
        low  |   .5557636   .0354425    15.68   0.000     .4858682     .625659
     middle  |   .5351441   .0331939    16.12   0.000      .469683    .6006051
       high  |   .5524211   .0359862    15.35   0.000     .4814536    .6233887
------------------------------------------------------------------------------

. qui reg female  read write i.prog

. margins, over(ses)

Predictive margins                                         Number of obs = 200
Model VCE: OLS

Expression: Linear prediction, predict()
Over:       ses

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
         ses |
        low  |   .5526417   .0363873    15.19   0.000     .4808786    .6244048
     middle  |   .5393238   .0337023    16.00   0.000     .4728559    .6057917
       high  |   .5481049   .0369571    14.83   0.000      .475218    .6209918
------------------------------------------------------------------------------

.
end of do-file

On the other hand, if you want expected mean values of y fully adjusted for all of the covariates including ses itself, and taking into account the fact that the distributions of read, write, and prog may differ across the ses categories, then you want to use -margins ses-. The key difference between -margins ses- and -margins, over(ses)- is that in -margins, over(ses)-, each ses's result is calculated using only the observations with that value of ses, whereas for -margins ses-, each ses's result is calculated by (temporarily) changing the value of ses to that particular value in the entire estimation sample, calculating predicted values of y, and then averaging over the entire data set. So, -margins ses- calculates the expected outcome values in each ses as if the distribution of all other model variables in the entire estimation sample prevailed in that ses group.

So what I have shown in my second code block here, and what -margins ses- gives you are two different things. Both are perfectly legitimate things to calculate, depending on what your purpose is. It boils down to whether you want the results to reflect the overall distribution of the covariates in the entire estimation sample, or whether you want each ses's results to reflect only its own distribution of the covariates.

[Weston Ley]

Thank you for the extremely informative post, Clyde. The project that I am working wants to account for the fact that the distribution of covariates is different for each level of our group variable. We essentially want to rerun our regression model for each level of our group variable while only using the sample that corresponds to that level. I believe this means that the margins, over(group) command is suitable for our project. However, I would like to know more about the following comment:

If what you want is ses-conditional mean values, crude, adjusted for read and write, and for read, write, and i.prog, what you have to do is omit i.ses from the regression.

Our research project is breaking down analysis of our outcome variable by race. Does this mean that calculating the average outcome value by race while controlling for other factors will require code that looks similar to this:

reg outcome x y z
margins, over(race)

Where x, y, and z are the factors that I want to control for?


Also, I am not sure if I understand the output to the commands:

qui reg female
margins, over(ses)

Why is the mean value of female 0.545 for every level of ses? Am I interpreting the output incorrectly?

[Clyde Schechter]

With -reg female-, there is nothing to distinguish any observation from any other. So xb will, for all observations, be the mean value of female. That is true whether calculated -over(ses)- or in the whole estimation sample.

If you run

reg female i.ses
margins, over(ses)

you will get the session specific means of female. You can verify that by comparing the results with the output of -tab ses female, row-

. qui: reg female i.ses

. margins, over(ses)

Predictive margins                                         Number of obs = 200
Model VCE: OLS

Expression: Linear prediction, predict()
Over:       ses

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
         ses |
        low  |   .6808511   .0723453     9.41   0.000     .5381804    .8235217
     middle  |   .5052632   .0508859     9.93   0.000     .4049122    .6056142
       high  |         .5   .0651246     7.68   0.000     .3715691    .6284309
------------------------------------------------------------------------------

. tab ses female, row

+----------------+
| Key            |
|----------------|
|   frequency    |
| row percentage |
+----------------+

           |        female
       ses |      male     female |     Total
-----------+----------------------+----------
       low |        15         32 |        47
           |     31.91      68.09 |    100.00
-----------+----------------------+----------
    middle |        47         48 |        95
           |     49.47      50.53 |    100.00
-----------+----------------------+----------
      high |        29         29 |        58
           |     50.00      50.00 |    100.00
-----------+----------------------+----------
     Total |        91        109 |       200
           |     45.50      54.50 |    100.00

And that's fine. But those numbers will then be preserved with any other covariates added in.

We essentially want to rerun our regression model for each level of our group variable while only using the sample that corresponds to that level. I believe this means that the margins, over(group) command is suitable for our project.

That is correct. Bear in mind that if you are doing a regression within a level of ses, you cannot adjust that regression for ses because ses doesn't vary.