marginal effects command with pwcompare

[Meng Yu]

After running a xtlogit regression, I was trying to compare the effects of investing on the mental health of men and women. So I used the command

margins sex, dydx (invest) pwcompare

. However, I am not sure how to interpret the results, as there seems to be no standard error or p value.

When the variable invest is continuous, the result seems to be the difference between the two derivatives using the command

margins, dydx (invest) at (sex=(0 1))

. But if the dependent variable is categorical, I don't know how to understand the result.

Thank you.

[Clyde Schechter]

However, I am not sure how to interpret the results, as there seems to be no standard error or p value.

Please show the output of your -xtlogit- and the complete output of -margins, sex, dydx(invest)-. In a similarly structured problem, I cannot replicate your finding:

. clear*

. webuse nlswork
(National Longitudinal Survey of Young Women, 14-24 years old in 1968)

. xtlogit union i.south##c.tenure, fe
note: multiple positive outcomes within groups encountered.
note: 2,899 groups (11,485 obs) omitted because of all positive or
      all negative outcomes.

Iteration 0:   log likelihood = -2784.4625  
Iteration 1:   log likelihood = -2775.5273  
Iteration 2:   log likelihood = -2775.5175  
Iteration 3:   log likelihood = -2775.5175  

Conditional fixed-effects logistic regression        Number of obs    =  7,522
Group variable: idcode                               Number of groups =  1,235

                                                     Obs per group:
                                                                  min =      2
                                                                  avg =    6.1
                                                                  max =     12

                                                     LR chi2(3)       =  81.87
Log likelihood = -2775.5175                          Prob > chi2      = 0.0000

--------------------------------------------------------------------------------
         union | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
---------------+----------------------------------------------------------------
       1.south |  -.9766129   .1829585    -5.34   0.000    -1.335205   -.6180208
        tenure |   .0682536   .0119506     5.71   0.000      .044831    .0916763
               |
south#c.tenure |
            1  |  -.0225043   .0191106    -1.18   0.239    -.0599605    .0149518
--------------------------------------------------------------------------------

. margins south, dydx(tenure)

Average marginal effects                                 Number of obs = 7,522
Model VCE: OIM

Expression: Pr(union|fixed effect is 0), predict(pu0)
dy/dx wrt:  tenure

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
tenure       |
       south |
          0  |   .0164573   .0026901     6.12   0.000     .0111849    .0217298
          1  |   .0097722   .0033196     2.94   0.003      .003266    .0162785
------------------------------------------------------------------------------

. margins south, dydx(tenure) pwcompare

Pairwise comparisons of average marginal effects

Model VCE: OIM                                           Number of obs = 7,522

Expression: Pr(union|fixed effect is 0), predict(pu0)
dy/dx wrt:  tenure

--------------------------------------------------------------
             |   Contrast Delta-method         Unadjusted
             |      dy/dx   std. err.     [95% conf. interval]
-------------+------------------------------------------------
tenure       |
       south |
     1 vs 0  |  -.0066851   .0041647     -.0148478    .0014776
--------------------------------------------------------------

.
end of do-file

The standard error of the difference is shown. The p-value is not, but you can get that with -margins sex, dydx(invest) pwcompare(effects)-.

[Meng Yu]

Thank you for your reply. I cannot have my result until it it released. So I must have remembered it wrongly. And now I know how to have p value too. Thank you.

For continuous variable, can I interpret the result as the difference between probabilities? In this case, for workers in the South vs not in the South, there is a 0.007 percentage points difference in their likelihood of being a union member as their tenure increases. Is that correct?

For categorical variables, how are we supposed to interpret the result?

Thank you.

[Clyde Schechter]

For continuous variable, can I interpret the result as the difference between probabilities? In this case, for workers in the South vs not in the South, there is a 0.007 percentage points difference in their likelihood of being a union member as their tenure increases. Is that correct?

For categorical variables, how are we supposed to interpret the result?

There seems to be some confusion here about the difference between continuous and categorical variables. The variable south in the example shown in #2 is a categorical variable, and your interpretation of it is correct: the probability of union is 0.007 higher among respondents with south = 1 than it is among south = 0.

For continuous variables it's a bit more complicated. The marginal effect of a continuous variable is a calculus construct: it is the partial derivative of the outcome probability with respect to the continuous predictor in general. In simpler terms it might be described as a rate of change in outcome probability per unit change in the continuous predictor, all else held constant. There is an additional complication as well since this is a nonlinear model. The rate of change in outcome probability per unit change in the predictor is not the same for all values of the predictor, because the logit transformation that relates the linear predictor of the model to the probability is non-linear. So, when you specify -margins sex, dydx(invest)- you are getting an average marginal effect, averaged over the observed distribution of invest in your data. But at any given level of invest, it will be different.

Now, if the marginal effect is fairly small, and if the data are falling mostly on the flat ends of the logistic curve, then the average marginal effect won't differ much from the marginal effects of invest at various observed levels, and it will also be reasonably accurate to refer to the marginal effect as the difference in outcome probability associated with a unit change in invest, eliding the difference between a difference and a differential. (To put this in concrete terms, the marginal effect is like an instantaneous speed. The speedometer says I'm driving at 40 km/hr right now. It does not follow, however, that in one hour I will travel 40 km because my speed may change as I go.) Given how many conditions have to hold in order for this looser language to not be misleading, it is probably safer to avoid it and stick with the always correct description in terms of rate of change of the outcome probability per unit change in invest.

[Meng Yu]

I think I meant tenure is a continuous variable. I am not sure how to interpret if both variables are categorical.

What you explained was in the situation when both variables are continuous and I am still trying to understand what you wrote. But I haven’t done any research when both variables in the interaction term are continuous.

[Clyde Schechter]

No, the example I showed involved an interaction between dichotomous variable south and continuous variable tenure. In that example, the outcome variable, union, is also dichotomous. So that example is, I believe, precisely analogous to what you are working on with sex corresponding to south, invest corresponding to tenure, and union corresponding to your undisclosed outcome variable (that has something to do with mental health).

Continuous # continuous interactions are pretty complicated to interpret, and I would advise you to stay away from them until you have enough experience under your belt with discrete # continuous interactions to feel completely comfortable with the latter.

[Meng Yu]

I meant in my first post that in addition to interpreting categorical #continuous, I am also interested in knowing how to interpret categorical # categorical when I use pwcompare to compare them. Thank you.

[Clyde Schechter]

Oh. The output of -margins, pwcompare- in #2 is interpreted as follows. The expected probability of union, averaged over the observed distribution of all other model variables is, to three decimal places, 0.007 lower when south = 1 than when south = 0. The 95% confidence interval for the difference is from 0.15 lower to 0.001 higher.

[Meng Yu]

What about tenure? Isn’t tenure a condition in this interpretation?

[Clyde Schechter]

Sorry, my mistake. Yes, the -margins, pwcompare- command involved dydx(tenure). So the interpretation is that the average rate of change in probability of union is 0.007 per year (tenure is measured in year units in that data set) lower when south = 1 than when south = 0.

That said, there is another issue that I failed to bring up from the very beginning, which is that the use of -margins- after -xtlogit, fe- is not appropriate. The problem is that the -xtlogit, fe- model is conditional on the fixed effects. The fixed effects themselves are not estimable in this model, but the margins and marginal effects depend on them. The use of the -predict(pu0)- is a workaround that gives results conditional on the fixed effects being zero--but we don't know what zero means as a fixed effect, it might be completely meaningless. So it is best to avoid the use of -margins- after -xtlogit, fe- in the first place. (The same is true for -xtpoisson, fe- and -xtnbreg, fe-.)

Not knowing enough about your project, I can't advise you about steps forward from here. You have said that the outcome variable is mental health. Most mental health variables are actually continuous anyway, so perhaps the solution is to use continuous outcome variables and use -xtreg, fe-, which does not condition on the fixed effects, and is compatible with the use of -margins- afterward. Or, if that isn't feasible, perhaps you should be using a random effects or population-averaged model.

[Meng Yu]

Thank you. And sorry for not making my project more clear. My dependent variable is dichotomous and I am using xtlogit random effects model. I have three independent variables in three separate models. Two of them are categorical. When I interact them with sex using pwcompare, I am not quite sure how to interpret the results.

[Clyde Schechter]

Well, if you don't see how to transfer what's shown in the examples I've given to your situation, I suggest you post your results so I can give you more direct advice.