Interpreting margins after Probit with DID: Heterogenous effects using Interaction terms

[Mehreen Mookerjee]

I’m estimating a difference-in-differences (DiD) model using a binary outcome (Y) and a policy treatment (treatpost), implemented via a Probit specification. The baseline model is:

probit Y treatpost post treat `Xvars' i.district, vce(cluster district)
margins, dydx(treatpost) pred(pr)

This works fine and gives us the average marginal effect of the DiD term (treatpost). Now, I would like to assess heterogeneous effects by age group — specifically comparing younger vs older women (binary variable younger, where 1 = younger).

To do this, I interacted treatpost with younger:

probit Y treatpost##i.younger post treat `Xvars' i.district, vce(cluster district)
margins, dydx(treatpost) at(younger=(0 1)) pred(pr) post

This gives me the marginal effect of treatpost for both younger = 0 and younger = 1. Both effects are positive and significant.

My main question is:

How do I formally test whether the marginal effects at younger = 0 and 1 are statistically different from each other? Also: Is there a better or more standard way to evaluate heterogeneous effects in a nonlinear DiD model like this, given the binary outcome?

Any guidance on the correct interpretation or alternative modeling strategies would be much appreciated.

[George Ford]

margins, dydx(treatpost) at(younger=(0 1)) pred(pr) post
lincom _b[2._at] - _b[1._at]

[Mehreen Mookerjee]

Thank you so much George!

We followed your advice and implemented the lincom command. However, we get that coefficient is 0 and std.error is omitted. Somehow the test is not working.
Is there something wrong in our main specification?

probit Y treatpost##i.younger post treat `Xvars' i.district, vce(cluster district)

where treatpost= treat x post

We have also tried by including all the double interactions but to no avail.

We would highly appreciate any advice and feedback on this. Thank you!

[George Ford]

What's being stored by margins is not _b[2._at] etc. Use coefleg in margins to get the actual names.

Run this for some guidance. The easier way is to use c. not i.

clear all

sysuse auto, clear

tab rep78
g treat = rep78>4

summ mpg
g himpg = mpg>22

## Let's see the coefficient by group
probit foreign himpg if treat
probit foreign himpg if !treat

## Joint for Test ; coef on interaction a direct test
probit foreign treat himpg c.treat#c.himpg
probit foreign c.treat##c.himpg  //same thing


## Using i ; now you have to test the difference, so previous model easier
probit foreign i.treat##i.himpg
margins, over(himpg) predict(pr) post coefleg
lincom _b[0bn.himpg] - _b[1.himpg]

[Mehreen Mookerjee]

Thank you so much. This works now.

[George Ford]

Interaction terms in non-linear models often require special interpretation. I'd look into that.

[George Ford]

And DID with non-linear models are also problematic. See Lechner's paper and some follow ups. Might be best to use LPM.