Using the margins command correctly

[Claude Francoeur]

Hello,

I am making this estimation:

label define Blev 0 "Low" 1 "High"
label values B Blev

reg Y c.A##c.A##i.B C, robust cformat(%6.2fc)


I was told to use the following commands to analyze the results. I need help to interpret the commands and the results from these commands.

margins B, at(A=(.12(0.10).72))

The results from this command show the predicted values of Y at different levels of A for both levels of B. Significant P-values mean the predicted values are different from zero. Correct?


margins r.B, at(A=(.12(0.10).72))

This command calculates the differences in the predicted values of the two levels of B and tests if these differences are significant. Correct?


margins B, dydx(A) at(A=(.12(0.10).72))

Can you help me understand what the command above does exactly and why it could be relevant?


margins r.B, dydx(A) at(A=(.12(0.10).72))

Same question here.


Please find the data below.

Thank you so much!

* Example generated by -dataex-. For more info, type help dataex
clear
input double(A Y) byte(B C)
.222222222222222            6.125 0  9
              .4 6.16666666666667 0 15
.466666666666667 6.30555555555556 1 15
.466666666666667            6.125 0 15
.357142857142857 6.86363636363636 1 15
              .4 6.43181818181818 1 15
.428571428571429 6.21428571428571 0 14
.583333333333333 6.45833333333333 1 11
.294117647058824 5.88888888888889 1 17
.461538461538462 6.54545454545454 1 13
            .375 6.41666666666667 1  8
.461538461538462          6.65625 0 13
.454545454545455 6.08333333333333 0 11
.545454545454545             6.25 0 11
           .3125              6.2 0 14
              .5             6.25 0  8
.571428571428571            6.425 0 14
.529411764705882 6.69230769230769 1 17
              .2 6.04166666666667 0 10
              .4             6.85 0 10
.571428571428571 6.32142857142857 0 14
              .5 6.52272727272727 1 16
.307692307692308           6.5625 1 13
.357142857142857 6.17857142857143 0 14
           .4375 6.55555555555556 0 16
              .5          6.40625 1 16
.454545454545455            6.375 0 11
.454545454545455          6.65625 1 11
            .375             6.75 1  8
.461538461538462 6.79545454545454 1 13
.444444444444444          6.40625 1  9
.428571428571429 6.57142857142857 1 14
.285714285714286              6.5 1  7
.285714285714286 6.58333333333333 1  7
.533333333333333            6.625 1 15
.333333333333333             6.25 1 15
              .5             5.45 0 10
.444444444444444              6.5 1  9
.466666666666667 6.41666666666667 0 15
              .5 6.47222222222222 1 14
              .5           6.3125 0 13
.333333333333333 6.11538461538461 0 15
.333333333333333 6.33333333333333 0 15
.416666666666667 6.36111111111111 0 12
.285714285714286              6.5 1  7
.545454545454545 6.69444444444444 1 11
.428571428571429 6.29166666666667 1  7
.428571428571429             6.75 1  7
.333333333333333 6.32142857142857 0 15
.454545454545455 6.41666666666667 0 11
.428571428571429 6.83333333333333 1  8
             .25                7 1  8
              .5 6.29166666666667 1 12
.333333333333333 6.17857142857143 0 12
              .5              6.7 0 10
.571428571428571                7 1  7
             .25              6.5 1 12
              .2             6.65 1 10
.363636363636364 6.54166666666667 1 11
.444444444444444          6.46875 0  9
           .4375             6.25 1 16
.384615384615385             6.25 0 13
.545454545454545 6.53571428571428 1 11
              .5             6.75 1 12
              .4          6.34375 0 15
.416666666666667            6.375 1 12
.428571428571429             6.55 1  7
.555555555555556             5.35 0  9
.416666666666667 6.91666666666667 1 12
.416666666666667             6.25 1 12
.357142857142857          6.59375 0 14
.529411764705882 6.52272727272727 1 17
.571428571428571 6.52777777777778 1 14
.363636363636364             6.45 1 11
.363636363636364 6.60714285714286 1 11
.545454545454545              6.5 1 11
.555555555555556              6.5 0  9
.555555555555556              6.7 1  9
.545454545454545            6.625 0 11
.181818181818182              5.8 0 11
.384615384615385            6.125 0 13
            .375              6.5 1 16
              .4             6.35 0 15
.533333333333333            6.375 0 15
              .5              6.5 1 16
.428571428571429 6.83333333333333 1  7
              .4              6.5 0  5
.466666666666667              6.5 1 15
              .5 6.28571428571428 1 12
.714285714285714 5.83333333333333 0  7
.153846153846154 6.55555555555556 1 13
.615384615384615 6.63888888888889 1 13
              .4          6.78125 1 15
.444444444444444             6.35 1  9
           .4375            6.225 0 16
.384615384615385          6.53125 0 15
.470588235294118 6.71153846153846 1 17
.538461538461538 6.89285714285714 0 13
.444444444444444             5.75 0  9
              .5            6.625 0 16
end

[Clyde Schechter]

margins B, at(A=(.12(0.10).72))

The results from this command show the predicted values of Y at different levels of A for both levels of B. Significant P-values mean the predicted values are different from zero. Correct?

Your statement of what the results show is correct. I disagree in general with this kind of interpretation of "significant p-values," but in the sense that this statement is commonly (mis-)used, it is correct.

margins r.B, at(A=(.12(0.10).72))

This command calculates the differences in the predicted values of the two levels of B and tests if these differences are significant. Correct?

Ditto.

margins B, dydx(A) at(A=(.12(0.10).72))

Can you help me understand what the command above does exactly and why it could be relevant?

This command will show you the marginal effect of A on Y at both values of B and the selected values of A. It is relevant because when you have an interaction between A and B, the marginal effect of A depends on the value of B and vice versa. In your case, A enters the interaction quadratically, so it is even more subtle: the marginal effect of A also depends on the value of A a which it is evaluated.

margins r.B, dydx(A) at(A=(.12(0.10).72))

Same question here.

This shows the differences between the marginal effect of A on Y when B = 1 and the marginal effect of A on Y when B = 0 at the selected values of A. Another way to say that is that it shows the effect of B on the marginal effect of A.

[Claude Francoeur]

Thank you Clyde.

Concerning the first two tables below.

The first table show that the marginal effect of A on Y at B=1 is never significantly different from zero (the coefficients are almost also always negative?). Correct?
I must not be interpreting this right, because the predicted values of Y are always "significantly" higher when B=1 (see the results from " margins r.B, at(A=(.12(0.10).72))

Can you help me interpret these results correctly?
.
. margins B, dydx(A) at(A=(.12(0.10).72))

Average marginal effects Number of obs = 111
Model VCE: Robust

Expression: Linear prediction, predict()
dy/dx wrt: A
1._at: A = .12
2._at: A = .22
3._at: A = .32
4._at: A = .42
5._at: A = .52
6._at: A = .62
7._at: A = .72

------------------------------------------------------------------------------
| Delta-method
| dy/dx std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
A |
_at#B |
1#Low | 3.458917 1.161161 2.98 0.004 1.156292 5.761542
1#High | -1.022182 .9271002 -1.10 0.273 -2.860657 .8162923
2#Low | 2.40869 .8202559 2.94 0.004 .7820917 4.035288
2#High | -.5735872 .624679 -0.92 0.361 -1.812349 .6651748
3#Low | 1.358463 .5233684 2.60 0.011 .3206036 2.396322
3#High | -.1249922 .3626132 -0.34 0.731 -.8440678 .5940834
4#Low | .3082356 .38933 0.79 0.430 -.4638205 1.080292
4#High | .3236028 .2886312 1.12 0.265 -.2487638 .8959693
5#Low | -.7419916 .5530009 -1.34 0.183 -1.838613 .35463
5#High | .7721977 .4956641 1.56 0.122 -.2107227 1.755118
6#Low | -1.792219 .8582604 -2.09 0.039 -3.494182 -.0902564
6#High | 1.220793 .7865196 1.55 0.124 -.3389051 2.780491
7#Low | -2.842447 1.201658 -2.37 0.020 -5.225379 -.4595146
7#High | 1.669388 1.096403 1.52 0.131 -.5048197 3.843596
------------------------------------------------------------------------------

Concerning the other test below. It seems the differences in marginal effects are always significant (at the 10% level), except for level 4.

Can you help me interpret these results correctly?
.
. margins r.B, dydx(A) at(A=(.12(0.10).72))

Contrasts of average marginal effects Number of obs = 111
Model VCE: Robust

Expression: Linear prediction, predict()
dy/dx wrt: A
1._at: A = .12
2._at: A = .22
3._at: A = .32
4._at: A = .42
5._at: A = .52
6._at: A = .62
7._at: A = .72

----------------------------------------------------
| df F P>F
-----------------+----------------------------------
A |
B@_at |
(High vs Low) 1 | 1 9.05 0.0033
(High vs Low) 2 | 1 8.40 0.0046
(High vs Low) 3 | 1 5.46 0.0214
(High vs Low) 4 | 1 0.00 0.9755
(High vs Low) 5 | 1 3.84 0.0528
(High vs Low) 6 | 1 6.27 0.0139
(High vs Low) 7 | 1 7.27 0.0082
Joint | 2 4.63 0.0119
|
Denominator | 104
----------------------------------------------------

------------------------------------------------------------------
| Contrast Delta-method
| dy/dx std. err. [95% conf. interval]
-----------------+------------------------------------------------
A |
B@_at |
(High vs Low) 1 | -4.481099 1.489193 -7.434225 -1.527974
(High vs Low) 2 | -2.982277 1.029146 -5.023111 -.941443
(High vs Low) 3 | -1.483455 .6347891 -2.742266 -.2246444
(High vs Low) 4 | .0153672 .4991446 -.9744553 1.00519
(High vs Low) 5 | 1.514189 .7731139 -.0189245 3.047303
(High vs Low) 6 | 3.013012 1.203593 .6262414 5.399782
(High vs Low) 7 | 4.511835 1.6739 1.192429 7.83124
------------------------------------------------------------------

.
Again, thank you!

[Clyde Schechter]

The first table show that the marginal effect of A on Y at B=1 is never significantly different from zero (the coefficients are almost also always negative?). Correct?
I must not be interpreting this right, because the predicted values of Y are always "significantly" higher when B=1 (see the results from " margins r.B, at(A=(.12(0.10).72))

You are, indeed, not interpreting things correctly. If the predicted values of Y are always "significantly" higher when B = 1, then that tells you that the marginal effect of B is positive at the selected values of A. It tells you nothing about the marginal effect of A at B = 1. So there is no inconsistency between your findings.

Concerning the other test below. It seems the differences in marginal effects are always significant (at the 10% level), except for level 4.

Can you help me interpret these results correctly?

The best thing you can do to interpret these results is to get out an opaque black marker and redact all the p-values from your output. They are only going to distract you from what is really going on. You have Y as a quadratic function of A, with an interaction with dichotomous variable B. The marginal effect of A on Y, within either value of B, is going to go from negative infinity to positive infinity when we consider A ranging over the entire real number line. (This is true of any quadratic variable in any regression model.) By continuity, there will always be some value of A for which the marginal effect is exactly zero. In fact, that value reflects the turning point of the parabola that a Y vs A graph would look like. For some range of values of A near that turning point, the marginal effect of A will necessarily be "non significant" and beyond that range the marginal effect of A will be "significant." The point is that this has nothing to do with your particular model or data: it is a general truth about quadratic models. So there is no point at looking at what is significant or not here. (I would argue that, more generally, there is hardly ever any point in looking at whether anything is "significant," but that is a long story and I won't digress on it any further here.) Now, in some data sets, the turning point occurs outside the range of the observed data. But in yours it occurs within the range of your observed data, confirming the wisdom of including the quadratic A term in your model.

What is particularly interesting in your data, and that you will never see by looking at p-values, is that when B = low, the marginal effect of A decreases as A increases, whereas when B = high, it increases. To get a real understanding of what is going on here you need to look at a graph. So

margins B, dydx(A) at (A = (.12(0.10).72))
marginsplot, xdimension(A)

and then see how this plays out in terms of predicted values of Y:

margins B, at(A = (.12(0.20).72))
marginsplot, xdimension(A)

[Claude Francoeur]

Thank you Clyde for these great explanations!

[Claude Francoeur]

Hello. Using the same dataset and this command: margins B, dydx(A) at(A=(.12(0.10).72)) I get these results:

Expression: Linear prediction, predict()
dy/dx wrt: A
1._at: A = .12
2._at: A = .22
3._at: A = .32
4._at: A = .42
5._at: A = .52
6._at: A = .62
7._at: A = .72

------------------------------------------------------------------------------
| Delta-method
| dy/dx std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
A |
_at#B |
1#Low | 3.458917 1.161161 2.98 0.004 1.156292 5.761542
1#High | -1.022182 .9271002 -1.10 0.273 -2.860657 .8162923
2#Low | 2.40869 .8202559 2.94 0.004 .7820917 4.035288
2#High | -.5735872 .624679 -0.92 0.361 -1.812349 .6651748
3#Low | 1.358463 .5233684 2.60 0.011 .3206036 2.396322
3#High | -.1249922 .3626132 -0.34 0.731 -.8440678 .5940834
4#Low | .3082356 .38933 0.79 0.430 -.4638205 1.080292
4#High | .3236028 .2886312 1.12 0.265 -.2487638 .8959693
5#Low | -.7419916 .5530009 -1.34 0.183 -1.838613 .35463
5#High | .7721977 .4956641 1.56 0.122 -.2107227 1.755118
6#Low | -1.792219 .8582604 -2.09 0.039 -3.494182 -.0902564
6#High | 1.220793 .7865196 1.55 0.124 -.3389051 2.780491
7#Low | -2.842447 1.201658 -2.37 0.020 -5.225379 -.4595146
7#High | 1.669388 1.096403 1.52 0.131 -.5048197 3.843596

I am trying to calculate the CIs. For example, for the first point (1#Low), I calculated the upper value of the CI like this: 3.4589 + (1.6596 X 1.1612) = 5.38 (but as shown above, the upper-value is 5.76)

I obtain the factor t* factor using: display invttail(104, .05) = 1.6596

What am I doing wrong?

Thank you for your help!

[Clyde Schechter]

Where you went wrong is:

display invttail(104, .05) = 1.6596

While the coverage probability for the entire CI is supposed to be 95% = 1 - .05, each tail separately covers only 0.025.

Thus:

. display 3.458917 + invttail(104, 0.025)*1.161161
5.7615428

[Claude Francoeur]

Yes. Totally. Thank you!

[Claude Francoeur]

Hi again,

Now, I want to calculate the CIs of the turning point and prove the calculation.

reg Y c.A##c.A##i.B C, robust cformat(%6.2fc)

nlcom -_b[A]/(2*_b[A#A])

_nl_1: -_b[A]/(2*_b[A#A])

------------------------------------------------------------------------------
Y | Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
_nl_1 | .4493494 .0390473 11.51 0.000 .3728181 .5258807
------------------------------------------------------------------------------


My question: Why are the calculated values not exactly equal to the CI values generated by nlcom?

. display .4493494 + invttail(104, 0.025) * .0390473
.52678166

. display .4493494 - invttail(104, 0.025) * .0390473
.37191714

Many thanks again!!

[Clyde Schechter]

-nlcom- does not use a t-distribution for its calculations; it uses a normal distribution. So:

. display .4493494 + invnormal(0.025) * .0390473
.3728181

.
. display .4493494 - invnormal(0.025) * .0390473
.5258807

[Claude Francoeur]

Got it! Thank you so much Clyde!