Marginal effects and Std. Errors of interactions after non-linear models

[Enrique Hernandez]

Dear all,

Following the advice provided by Vince Wiggins in a post in the old statalist I am estimating marginal effects and their standard errors for a continuos*categorical interaction in a logistic model using the contrast feature of the margins command and I am comparing the results I obtain with the estimates obtained using the predictnl command as suggested by Karaca-Mandic et al (2012). The problem is that while the estimate of the "size" of the interaction effect is the same the standard errors that I obtain are different.

Here you have my code and results (I am replicating some of the analyses of Karaca-Mandic, Norton and Dowd (2012) paper "Interaction Terms in Nonlinear Models" published in the Health Services Research journal)

. webuse margex, clear
(Artificial data for margins)

. generate female = (sex==1)

. generate agefem = age*female

.
. * Using margins contrast feature
. quietly: logit outcome c.age##i.female, nolog

. margins r.female, dydx(age)  

Contrasts of average marginal effects
Model VCE    : OIM

Expression   : Pr(outcome), predict()
dy/dx w.r.t. : age

------------------------------------------------
             |         df        chi2     P>chi2
-------------+----------------------------------
age          |
      female |          1        9.61     0.0019
------------------------------------------------

--------------------------------------------------------------
             |   Contrast Delta-method
             |      dy/dx   Std. Err.     [95% Conf. Interval]
-------------+------------------------------------------------
age          |
      female |
   (1 vs 0)  |   .0040705    .001313       .001497    .0066439
--------------------------------------------------------------

.
. * Using predictnl
. quietly: logit outcome age female agefem, nolog

. predictnl phat = (_b[age]+_b[agefem])* ///
>         (1/(1+exp(-(_b[_cons]+_b[age]*age+_b[female]+_b[agefem]*age))))* ///
>         (1-(1/(1+exp(-(_b[_cons]+_b[age]*age+_b[female]+_b[agefem]*age))))) ///
>         -_b[age]*(1/(1+exp(-(_b[_cons]+_b[age]*age))))* ///
>         (1-(1/(1+exp(-(_b[_cons]+_b[age]*age))))), se(phat_se)

.        
. sum phat*

    Variable |        Obs        Mean    Std. Dev.       Min        Max
-------------+---------------------------------------------------------
        phat |      3,000    .0040705    .0022295  -.0022299   .0069318
     phat_se |      3,000    .0016307    .0012906   .0003856   .0039043

.
end of do-file

My questions are the following:

1) Why do I obtain a different result if I estimate the standard errors of the interaction using the contrast features in margins (Std. Err = .001313) than if I do it with the predictnl command (Std. Err = .0016307)?

My hunch is that the difference is due to the fact that the standard errors estimated by margins are Delta-method standard errors while predictnl estimates a different kind of standard errors, but I am not completely sure.

2) Given these differences what is the best (most accurate) way of estimating the standard errors of this type of interactions (margins or predictnl)?

Thank you very much

Enrique

[Maarten Buis]

The predictnl command also uses the delta method to compute the standard error, but the standard error of a mean is not the mean of standard errors (you are ignoring the covariances). With summarize you show the mean of the standard errors, so that is the problem.

Knowing Norton I suspect that his main point is that there is not one marginal effect of the interaction term in non-linear models but many. So you don't report the mean, but plot all the interaction effects, one for each observation in your data, and their confidence interval. It is not what I would do, as Vince already mentioned in the post you referenced in your question.

[Enrique Hernandez]

Dear Maarten,

Thanks for your answer, that explains it.

Enrique

[Moumie Maoulidi]

Hi Enrique and Maarten
Thanks for this interesting discussion. I have a follow-up question. Suppose you have 2 discrete variables x1 and x2

If one then wanted to put the marginal effects in a table using estate, one would type
margins, dydx(*) post asobserved vsquish vce(svy)
eststo m
esttab m using "path/filename.csv", replace se(%12.2fc) scalars(F) compress varwidth(45) label nonumbers nobaselevels interaction("X")

but the table that is generated only shows the marginal effects of the two terms. It does not show the marginal effect of the interaction term.

As Vince suggest, you can get the marginal effect of the interaction by

margins r.x1, dydx(x2) asobserved vsquish vce(svy)

and you can see it if you type

matrix list e(b)
di _b[1.x1:r1vs0.x2]
di _se[1.x1:r1vs0.x2]

What is the syntax to produce a table that has the marginal effects of x1, x2 and x1#x2? Thanks

[Moumie Maoulidi]

Hi Enrique and Maarten
Thanks for this interesting discussion. I have a follow-up question. Suppose you have 2 discrete variables x1 and x2

If one then wanted to put the marginal effects in a table using esttab, one would type
margins, dydx(*) post asobserved vsquish vce(svy)
eststo m
esttab m using "path/filename.csv", replace se(%12.2fc) scalars(F) compress varwidth(45) label nonumbers nobaselevels interaction("X")

but the table that is generated only shows the marginal effects of the two terms. It does not show the marginal effect of the interaction term.

As Vince suggest, you can get the marginal effect of the interaction by

margins r.x1, dydx(x2) asobserved vsquish vce(svy)

and you can see it if you type

matrix list e(b)
di _b[1.x1:r1vs0.x2]
di _se[1.x1:r1vs0.x2]

What is the syntax to produce a table that has the marginal effects of x1, x2 and x1#x2? Thanks

[Moumie Maoulidi]

Dear All
Since margins, dydx(*) calculates average marginal effects but does not provide marginal effects for interactions, one idea is to first of all calculate the average marginal effects, store the results, then calculate the marginal effect of the interaction term [margins r.x1, dydx(x2)], store the results in a matrix, and then combine the two stored results. Below is an example. First, 2 discrete variables (dum and w) are created.

eststo clear
sysuse auto, clear
set seed 12345
generate dum=uniform()>0.5
gen w=1 if weight>2930
replace w=0 if w !=1

xi: probit foreign w#i.dum turn, nolog
eststo: margins, dydx(*) post asobserved vsquish

xi: probit foreign w#i.dum turn, nolog
margins r.dum, dydx(w) post asobserved vsquish
estadd matrix Matr1
eststo Matr1

esttab est1 Matr1 using "/Volumes/chec0.csv", replace se(%12.2fc) compress varwidth(45) /*
label nonumbers nobaselevels interaction("X")

Does anyone have any other suggestions? For instance, I am still unsure how to add significance stars on the marginal effect of the interaction. Thanks
Moumie

[Maarten Buis]

There are just too many conceptual problems with marginal effects for interaction terms, so my suggestion is to not use marginal effects but estimate a logit model so you can simply interpret your model in terms of odds ratios: http://maartenbuis.nl/publications/interactions.html.

[Clyde Schechter]

I would just emphasize what Maarten said. There is no statistic you can calculate for an interaction term that is truly analogous to the marginal effect of a predictor variable. It simply doesn't exist. There are statistics that look look somewhat like marginal effects of interaction terms when written in differential calculus notation, but they could not be understood or used in the same way as marginal effects of predictor variables. You really just should not go there.

[Moumie Maoulidi]

Hi Maarten
I read your article in 2014 and found it very illuminating. Given the preference for probit over logit in economics and education, I was previously hesitant about using odds ratio. It is true that they have been given a bad reputation, except in health economics. Your suggestion and Clyde's are on point. If memory serves, Prof. Richards has also made a similar observation. Thanks again
Moumie