Extraordinary large treatment effect for a binary outcome variable using ivregress

[Juliane Z]

Dear Statalist,

I try to estimate a quasi-treatment effect by gender for a binary outcome variable in a fuzzy RDD set-up. I have two data waves (before treatment and after treatment). What I have estimated so far is roughly following Becker, Egger, Ehrlich (2013), (for simplicity I ignore the bandwidths below):

probit T  M  A  MxA  S , robust
predict Z  
gen MxZ = M*Z
ivregress 2sls  dY   M   S   (T   MxT  =   Z  MxZ)  , first vce(cluster ID)

Y = Outcome (binary)
dY = Difference in Y over time (Follow-up - Baseline)
T = Treatment (binary)
M = Male (binary)
A = Assignment to treatment group (binary, equals 1 if S > a certain threshold)
S = Score used for assignment
Z = predicted instrument (to avoid "forbidden regression")

Here are my concerns:
1. I am particularly interested in the estimated treatment effect for women, which would be the coefficient of T in the second stage output. The estimated coefficient is roughly -0.2. However, in outcome Y at baseline only 7% of the sample equal 1, while the rest equals 0. My interpretation of the coefficient usually would be that the treatment led to a reduction in Y of 20 percentage points over time compared to the quasi-control group. Yet, this obviously seems incorrect as the baseline probability of Y is only 7% to start from.

2. I did also consider ivprobit:

ivprobit  dY   M   S   (T   MxT  =   Z  MxZ)  , first vce(cluster ID)
margins, dydx(*)

But there the coefficient for the treatment effect is even larger (-0.7). I tried to use the margins post-estimation command, but the output for the marginal effect exactly equals the size of the coefficient, which also does not seem correct.

Did I go wrong with my implementation in Stata somewhere? Advice would be much appreciated.

Many thanks
Juliane

[William Lisowski]

Allow me to make what may be a naive comment regarding your second example, using ivprobit and margins.

If I understand you correctly, you believe that the margins command has reported a treatment effect of -0.7 which, it seems to me, you are interpreting as a 70 percentage-point change in the probability of your outcome (based on your reasoning in your second example0. But my understanding from help ivprobit postestimation is that for ivprobit estimation it is the case that margins reports by default the change in the linear prediction (xb) and requires the predict(pr) option to report the change in probability.

[Juliane Z]

Dear William,

thanks so much for this comment, it was actually right to the point. Once I applied the option correctly the effect size dropped to 1 to 2 percentage points which is also in line with descriptively comparing the treatment and control group.

I guess, this is proof that my example is one of the cases where the linear probability model infact does not resemble a probit/logistic regression. I assume that it is correct to proceed with the probit model.

Thanks again,
Juliane