Non linear difference-in-difference.

[Sumedha Gupta]

Dear All,

I have an individual level panel data with 58 monthly waves per individual. The dependent variable is incidence of poisoning (binary) and I am interested in evaluating the impact of an emergency department intervention on future poisonings (again binary). Since, my outcome variable is a binary, I want to implement Puhani 2010 (https://www.sciencedirect.com/scienc...65176511004769) non-linear changes-in-change model. I know that if treatment was at a fixed time for the whole sample, i.e. one pre and one post period only, I should do the following:

gen tp=treated*post
where, treated is a binary and post is also a binary and fixed across individuals. In this case, tp uniquely captures the interaction term and is 1 for the treated group following the intervention.Next,
probit y i.(treated post tp) x
margins treated, dydx(post) pwcompare(cimargins effects)

But, I am not sure how to implement this when the post variable varies across individuals (time varying treatment). In that case, how do I need to adapt the above code please?

Many thanks,
Sumedha.

[Phil Bromiley]

You didn't get a quick answer. Your question seems to require us read a paper to help you. This sometimes happens, but depends on the topic piquing the interest of one of the folks who respond on this list.

You will increase your chances of a helpful answer by following the FAQ on asking questions. I'm not sure why you don't just use factor variable notation instead of creating your interaction manually. This also makes it easier to interpret later.

[Sumedha Gupta]

Thank you Prof. Bromiley. Point well taken. Let me make a better effort to present my research problem and hopefully it may pique a contributors interest. So here it goes...

I have a sample of 558 distinct individuals observed of availing different health services within a health system over a period of almost 5 years (2013-2018). What lands them in the sample is that at least once over the observation window they were treated in the health system's emergency department for a drug overdose. So everybody in the panel has overdoses at least once. 70 of these people (12.54%), at different points in time, were recruited into a special program for addictions therapy and I am trying to evaluate the effectiveness of this program on incidences of repeat drug overdose.

The data itself is observational and for analyses I have converted it into an individual level panel with months as waves. So for an observation window of almost 5 years I have a panel for 558 persons over 58 monthly waves. Of course my panel is unbalanced as these individuals don't necessarily interact with the health system every month. Also, my outcome variable of repeat overdoses are small counts and in fact can also be recoded to binary. Here is the tab of the counts:

opioidpois | Freq. Percent Cum.
------------+-----------------------------------
0 | 1,296 79.85 79.85
1 | 315 19.41 99.26
2 | 11 0.68 99.94
3 | 1 0.06 100.00
------------+-----------------------------------
Total | 1,623 100.00

Since, for less than .75% of the sample counts are >1 I recode those to 1 as well. So I have a binary indicator for the outcome opioidpois, binary for the assignment to treatment(treat∈{0,1}), implementation time of treatment varies across individuals (post∈{0,1} when treat==1 with post=0 in the pre-perioid and 1 after treatment begins, post=0 always for the non treated i.e. treat==0) and the panel is completely unbalanced. To balance the panel I use :

tsset studypersonid month
tsfill, full
recode opioidpois .=0

So I have artifically balanced the panel with 0 incidence of poisonings. This is probably ok but it means there are a lot of 0s now. But I think they have the write interpretation as the 0 does mean that the person did not overdose that month. The opioidpois outcome variable for the balanced panel now tabulates like the following :

opioidpois | Freq. Percent Cum.
------------+-----------------------------------
0 | 28,978 98.88 98.88
1 | 327 1.12 100.00
------------+-----------------------------------
Total | 29,305 100.00


First, unlike the linear model where we need to confirm same pre-treatment trend assumption in outcome variable between teh treated and control group in the pre-tratment perioid I am not sure how to check for the underlying assumption in teh non-linear case. I don't even know what that equation should look like.

xtlogit opioidpois i.month## i.post##i.treat if post==0, fe vce (bootstrap)

Does this look right?

Assuming that holds, next, to measure the impact of the addiction therapy program with my binary outcome data I run:


xtset studypersonid month
xtlogit opioidpois i.month i.post##i.treat, fe vce(bootstrap)


Output I get is:

note: 1.post#0.treat identifies no observations in the sample
note: 1.post#1.treat omitted because of collinearity
(running xtlogit on estimation sample)

Bootstrap replications (50)
1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5
.................................................. 50

Conditional fixed-effects logistic regression Number of obs = 14,041
Group variable: studypersonid Number of groups = 237

Obs per group:
min = 37
avg = 59.2
max = 60

Wald chi2(42) = .
Log likelihood = -923.9404 Prob > chi2 = .

(Replications based on 237 clusters in studypersonid)

Observed Bootstrap Normal-based
opioidpois Coef. Std. Err. z P>z [95% Conf. Interval]

month
4 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
5 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
6 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
7 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
8 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
9 15.60325 8.304257 1.88 0.060 -.6727941 31.87929
10 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
11 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
12 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
13 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
14 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
15 16.29882 6.28325 2.59 0.009 3.983875 28.61376
16 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
17 15.60325 8.198561 1.90 0.057 -.4656346 31.67213
18 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
19 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
20 16.29882 5.132369 3.18 0.001 6.239561 26.35808
21 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
22 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
23 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
24 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
25 15.60325 7.934701 1.97 0.049 .0515227 31.15498
26 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
27 -1.34e-07 3.09e-07 -0.43 0.665 -7.40e-07 4.72e-07
28 15.60325 8.05699 1.94 0.053 -.18816 31.39466
29 15.60325 7.942865 1.96 0.049 .0355205 31.17098
30 16.29882 7.250728 2.25 0.025 2.087653 30.50999
31 16.70671 2.576896 6.48 0.000 11.65609 21.75733
32 16.70671 .608332 27.46 0.000 15.5144 17.89902
33 17.40717 .6658509 26.14 0.000 16.10213 18.71222
34 17.22241 .8138187 21.16 0.000 15.62735 18.81746
35 17.40717 .7474373 23.29 0.000 15.94222 18.87212
36 17.40717 .741802 23.47 0.000 15.95327 18.86108
37 17.12312 2.59812 6.59 0.000 12.0309 22.21535
38 18.3088 .6681007 27.40 0.000 16.99935 19.61825
39 18.14934 .6716754 27.02 0.000 16.83288 19.4658
40 18.83994 .5792557 32.52 0.000 17.70462 19.97526
41 18.36724 .6363181 28.86 0.000 17.12008 19.6144
42 18.49336 .6674329 27.71 0.000 17.18522 19.80151
43 18.68435 .677677 27.57 0.000 17.35613 20.01257
44 18.9268 .603721 31.35 0.000 17.74353 20.11008
45 18.76939 .6430381 29.19 0.000 17.50906 20.02972
46 18.83173 .6101603 30.86 0.000 17.63584 20.02763
47 19.07445 .6007763 31.75 0.000 17.89695 20.25195
48 17.87142 .5864062 30.48 0.000 16.72208 19.02075
49 17.72914 .7240905 24.48 0.000 16.30995 19.14834
50 16.25673 5.538684 2.94 0.003 5.401104 27.11235
51 17.8417 .6335004 28.16 0.000 16.60006 19.08334
52 17.21422 .6495983 26.50 0.000 15.94103 18.4874
53 17.05807 2.573263 6.63 0.000 12.01456 22.10157
54 16.34176 5.944654 2.75 0.006 4.690449 27.99307
55 17.39016 .715802 24.29 0.000 15.98722 18.79311
56 17.64531 .6308082 27.97 0.000 16.40895 18.88167
57 16.78773 3.502776 4.79 0.000 9.922412 23.65304
58 16.78999 4.736667 3.54 0.000 7.506295 26.07369
59 -.0508244 .0652461 -0.78 0.436 -.1787044 .0770557
60 -.007494 .0253488 -0.30 0.768 -.0571768 .0421888
61 16.27965 5.545031 2.94 0.003 5.411586 27.14771
62 15.68962 7.894522 1.99 0.047 .2166373 31.1626

1.post 3.223785 .460151 7.01 0.000 2.321905 4.125664
1.treat 0 (omitted)

post#treat
1 0 0 (empty)
1 1 0 (omitted)


My understanding is that given the non -linear setting the treatment effect is not the coefficient of the post variable but instead the change in marginal effect between the treated and those not (refer an earlier post https://www.statalist.org/forums/for...rences-model):

margins treat, dydx(post) pwcompare (cimargins effect)

But then I get:
Pairwise comparisons of average marginal effects
Model VCE : Bootstrap

Expression : Pr(opioidpois|fixed effect is 0), predict(pu0)
dy/dx w.r.t. : 1.post

--------------------------------------------------------------
| Delta-method Unadjusted
| Margin Std. Err. [95% Conf. Interval]
-------------+------------------------------------------------
1.post |
treat |
0 | . (not estimable)
1 | . (not estimable)
--------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the
base level.

------------------------------------------------------------------------------
| Contrast Delta-method Unadjusted Unadjusted
| dy/dx Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.post |
treat |
1 vs 0 | . (not estimable)
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.


Nothing?

I will appreciate any help I can get to understand how to run this. I will grateful for any input.
Thank you for your time and hoping to hear back.
Sumedha.

[Clyde Schechter]

If you run

xtlogit opioidpos i.post##i.treat i.month, fe

the coefficient of post#treat will be the generalized difference in differences (GDID) estimator of the treatment effect in the log-odds metric. (Because not all of your participants begin treatment at the same time, you do not have a classic DID model, so we use GDID instead.)

Pouhani, as I understand his paper, argues that this is the estimator you should use and he opposes trying to express the effect in the probability metric (i.e. based on marginal effect differences). I disagree (pretty strongly) with Pouhani about this, but I don't want to revisit that argument here. If you want to do it using marginal effects (which I do recommend) then you can run

margins treat, dyx(post) pwcompare noestimcheck

(Again, I recommend doing this and using it as the primary result of your study, but Pouhani would say it is wrong.)

The -noestimcheck- option will get you past the "not estimable" results. You should not routinely use -noestimcheck- in any model that produces "not estimable" results, but in the case of a model like this one where the problem has to do with the absorbed fixed effects and the fact that treat is colinear with them, it is OK to do that.

I'm not commenting here on issues of vce(cluster) or vce(bootstrap). I have no opinion to express about that.

[Sumedha Gupta]

Dear Prof. Schechter,
Thank you so much for your very helpful advice. Apologies for the delay in responding, I was traveling for work and had limited connectivity. I agree with your recommendation of the marginal effect. I know you mentioned that you do not want to revisit the argument of why this is a more correct estimate for the treatment effect but can you please direct me to the thread where you discussed it earlier. I found this one: https://www.statalist.org/forums/for...fference-model

Is this the one you were referring to?

Per you advice I ran the margins command and got the following:

. xtlogit opioidpois i.month i.post##i.treat, fe vce(bootstrap)
note: 1.post#0.treat identifies no observations in the sample
note: 1.post#1.treat omitted because of collinearity
(running xtlogit on estimation sample)

Bootstrap replications (50)
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5
.................................................. 50

Conditional fixed-effects logistic regression Number of obs = 14,041
Group variable: studypersonid Number of groups = 237

Obs per group:
min = 37
avg = 59.2
max = 60

Wald chi2(42) = .
Log likelihood = -923.9404 Prob > chi2 = .

(Replications based on 237 clusters in studypersonid)
------------------------------------------------------------------------------
| Observed Bootstrap Normal-based
opioidpois | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
month |
4 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
5 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
6 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
7 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
8 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
9 | 15.60325 7.865912 1.98 0.047 .1863463 31.02015
10 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
11 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
12 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
13 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
14 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
15 | 16.29882 6.302538 2.59 0.010 3.946071 28.65157
16 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
17 | 15.60325 8.116929 1.92 0.055 -.305639 31.51214
18 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
19 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
20 | 16.29882 5.601991 2.91 0.004 5.319119 27.27852
21 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
22 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
23 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
24 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
25 | 15.60325 8.138674 1.92 0.055 -.3482583 31.55476
26 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
27 | -1.84e-07 2.75e-07 -0.67 0.504 -7.23e-07 3.55e-07
28 | 15.60325 7.374684 2.12 0.034 1.149136 30.05736
29 | 15.60325 7.816178 2.00 0.046 .2838231 30.92268
30 | 16.29882 5.632029 2.89 0.004 5.260246 27.33739
31 | 16.70671 1.243258 13.44 0.000 14.26997 19.14345
32 | 16.70671 3.423502 4.88 0.000 9.996772 23.41665
33 | 17.40717 1.164923 14.94 0.000 15.12396 19.69038
34 | 17.22241 1.156357 14.89 0.000 14.95599 19.48883
35 | 17.40717 1.148221 15.16 0.000 15.1567 19.65764
36 | 17.40717 1.204355 14.45 0.000 15.04668 19.76767
37 | 17.12312 1.255803 13.64 0.000 14.6618 19.58445
38 | 18.3088 1.037241 17.65 0.000 16.27585 20.34175
39 | 18.14934 1.070194 16.96 0.000 16.0518 20.24688
40 | 18.83994 1.079422 17.45 0.000 16.72431 20.95557
41 | 18.36724 1.047899 17.53 0.000 16.3134 20.42108
42 | 18.49336 1.008975 18.33 0.000 16.51581 20.47092
43 | 18.68435 1.021974 18.28 0.000 16.68132 20.68738
44 | 18.9268 1.126724 16.80 0.000 16.71847 21.13514
45 | 18.76939 1.098917 17.08 0.000 16.61556 20.92323
46 | 18.83173 1.12355 16.76 0.000 16.62962 21.03385
47 | 19.07445 1.091935 17.47 0.000 16.93429 21.2146
48 | 17.87142 1.091265 16.38 0.000 15.73258 20.01026
49 | 17.72914 1.071032 16.55 0.000 15.62996 19.82833
50 | 16.25673 3.577862 4.54 0.000 9.244244 23.26921
51 | 17.8417 1.17372 15.20 0.000 15.54125 20.14215
52 | 17.21422 1.076638 15.99 0.000 15.10404 19.32439
53 | 17.05807 1.076036 15.85 0.000 14.94907 19.16706
54 | 16.34176 5.276126 3.10 0.002 6.00074 26.68278
55 | 17.39016 1.109467 15.67 0.000 15.21565 19.56468
56 | 17.64531 1.184504 14.90 0.000 15.32373 19.96689
57 | 16.78773 4.827053 3.48 0.001 7.326877 26.24858
58 | 16.78999 4.829097 3.48 0.001 7.325136 26.25485
59 | -.0508244 .0646349 -0.79 0.432 -.1775065 .0758577
60 | -.007494 .0182666 -0.41 0.682 -.0432959 .0283078
61 | 16.27965 6.293383 2.59 0.010 3.944844 28.61445
62 | 15.68962 8.341851 1.88 0.060 -.6601109 32.03934
|
1.post | 3.223785 .5645566 5.71 0.000 2.117274 4.330295
1.treat | 0 (omitted)
|
post#treat |
1 0 | 0 (empty)
1 1 | 0 (omitted)
------------------------------------------------------------------------------

. margins treat, dydx(post) pwcompare noestimcheck

Pairwise comparisons of average marginal effects
Model VCE : Bootstrap

Expression : Pr(opioidpois|fixed effect is 0), predict(pu0)
dy/dx w.r.t. : 1.post

--------------------------------------------------------------
| Contrast Delta-method Unadjusted
| dy/dx Std. Err. [95% Conf. Interval]
-------------+------------------------------------------------
1.post |
treat |
1 vs 0 | 0 (omitted)
--------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the
base level.

Again nothing?

Also, sorry for asking more, but how would one check the equality of the pre-treatment trend assumption in the case of a non-linear GDID? I am thinking:

xtlogit opioidpois i.month##i.post##i.treat if post==0, fe

and then, margins again?

Thank you so much Prof. Schechter. I am very grateful for your advise.

Sumedha.

[Clyde Schechter]

Like you, I've been traveling with limited connectivity the last few days.

As for the Puhani issue, the link you found was one of the threads I have in mind. There was another one that was mostly a dialog between me and Joseph Coveney, but I cannot seem to find the link to it now.

As for your marginal effect estimation, the key to the problem is in the following piece of your output:

note: 1.post#0.treat identifies no observations in the sample

You have one of those situations where different members of the treated group undergo treatment at different time points, so you have just set your post variable to 0 at all times for the non-treated group. So this leads to the interaction term treat#post being colinear with the individual and time fixed effects, so it gets omitted and your DID estimate goes up in smoke. I should have noticed this before my earlier response to your post. In this situation you cannot do a classical DID estimate. You can do generalized DID instead. For this, you do not use the treat and post variables. Instead you create a variable, let's call it received_treatment and set it to 1 in the post-treatment observations of the treatment group, and 0 everywhere else. Your model is then:

xtlogit opioidpois i.month i.under_treatment, fe

and the marginal effect would be gotten with -margins under_treatment-

Notes:
1. Again I'm not commenting on your choice of bootstrap standard errors.
2. For this model, the inclusion of the i.month variables is not optional. They must be there in order for under_treatment to provide a (generalized) DID estimate. If you were to leave them out, your results would be invalid for more profound reasons than just failure to adjust for monthly shocks. That is, even if you were confident that there are no important monthly shocks to your outcome, you will still need the i.month variables here.

For assessing parallel trends before treatment, I would do:

xtlogit opioidpois i.month##i.treat if post == 0, fe  // not i.month##i.post##i.treat
margins month#treat
marginsplot

[Sumedha Gupta]

Dear Prof. Schechter,

Thank you for the correct model and code. I ran it calling the variable that you called under_treatment *did*. I got the following:

xtlogit opioidpois i.month i.did, fe
note: multiple positive outcomes within groups encountered.
note: 258 groups (15,264 obs) dropped because of all positive or
all negative outcomes.

Iteration 0: log likelihood = -948.39567
Iteration 1: log likelihood = -927.96701
Iteration 2: log likelihood = -925.00118
Iteration 3: log likelihood = -924.13745
Iteration 4: log likelihood = -923.98637
Iteration 5: log likelihood = -923.95108
Iteration 6: log likelihood = -923.94267
Iteration 7: log likelihood = -923.94088
Iteration 8: log likelihood = -923.94051
Iteration 9: log likelihood = -923.94042
Iteration 10: log likelihood = -923.9404

Conditional fixed-effects logistic regression Number of obs = 14,041
Group variable: studypersonid Number of groups = 237

Obs per group:
min = 37
avg = 59.2
max = 60

LR chi2(60) = 665.24
Log likelihood = -923.9404 Prob > chi2 = 0.0000

------------------------------------------------------------------------------
opioidpois | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
month |
4 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
5 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
6 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
7 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
8 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
9 | 15.60325 2441.622 0.01 0.995 -4769.887 4801.094
10 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
11 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
12 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
13 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
14 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
15 | 16.29882 2441.622 0.01 0.995 -4769.192 4801.789
16 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
17 | 15.60325 2441.622 0.01 0.995 -4769.887 4801.094
18 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
19 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
20 | 16.29882 2441.622 0.01 0.995 -4769.192 4801.789
21 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
22 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
23 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
24 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
25 | 15.60325 2441.622 0.01 0.995 -4769.887 4801.094
26 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
27 | -1.95e-07 3452.974 -0.00 1.000 -6767.705 6767.705
28 | 15.60325 2441.622 0.01 0.995 -4769.887 4801.094
29 | 15.60325 2441.622 0.01 0.995 -4769.887 4801.094
30 | 16.29882 2441.622 0.01 0.995 -4769.192 4801.789
31 | 16.70671 2441.622 0.01 0.995 -4768.784 4802.197
32 | 16.70671 2441.622 0.01 0.995 -4768.784 4802.197
33 | 17.40717 2441.622 0.01 0.994 -4768.083 4802.898
34 | 17.22241 2441.622 0.01 0.994 -4768.268 4802.713
35 | 17.40717 2441.622 0.01 0.994 -4768.083 4802.898
36 | 17.40717 2441.622 0.01 0.994 -4768.083 4802.898
37 | 17.12312 2441.622 0.01 0.994 -4768.367 4802.614
38 | 18.3088 2441.622 0.01 0.994 -4767.182 4803.799
39 | 18.14934 2441.622 0.01 0.994 -4767.341 4803.64
40 | 18.83994 2441.622 0.01 0.994 -4766.65 4804.33
41 | 18.36724 2441.622 0.01 0.994 -4767.123 4803.858
42 | 18.49336 2441.622 0.01 0.994 -4766.997 4803.984
43 | 18.68435 2441.622 0.01 0.994 -4766.806 4804.175
44 | 18.9268 2441.622 0.01 0.994 -4766.564 4804.417
45 | 18.76939 2441.622 0.01 0.994 -4766.721 4804.26
46 | 18.83173 2441.622 0.01 0.994 -4766.659 4804.322
47 | 19.07445 2441.622 0.01 0.994 -4766.416 4804.565
48 | 17.87142 2441.622 0.01 0.994 -4767.619 4803.362
49 | 17.72914 2441.622 0.01 0.994 -4767.761 4803.22
50 | 16.25673 2441.622 0.01 0.995 -4769.234 4801.747
51 | 17.8417 2441.622 0.01 0.994 -4767.649 4803.332
52 | 17.21422 2441.622 0.01 0.994 -4768.276 4802.705
53 | 17.05807 2441.622 0.01 0.994 -4768.432 4802.548
54 | 16.34176 2441.622 0.01 0.995 -4769.149 4801.832
55 | 17.39016 2441.622 0.01 0.994 -4768.1 4802.881
56 | 17.64531 2441.622 0.01 0.994 -4767.845 4803.136
57 | 16.78773 2441.622 0.01 0.995 -4768.703 4802.278
58 | 16.78999 2441.622 0.01 0.995 -4768.7 4802.28
59 | -.0508244 3464.816 -0.00 1.000 -6790.966 6790.864
60 | -.007494 3511.808 -0.00 1.000 -6883.025 6883.01
61 | 16.27965 2441.622 0.01 0.995 -4769.211 4801.77
62 | 15.68962 2441.622 0.01 0.995 -4769.801 4801.18
|
1.did | 3.223785 .4633592 6.96 0.000 2.315617 4.131952
------------------------------------------------------------------------------

.
end of do-file

. margins did

Predictive margins Number of obs = 14,041
Model VCE : OIM

Expression : Pr(opioidpois|fixed effect is 0), predict(pu0)

------------------------------------------------------------------------------
| Delta-method
| Margin Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
did |
0 | .815095 220.2242 0.00 0.997 -430.8163 432.4465
1 | .9858264 32.51139 0.03 0.976 -62.73533 64.70698
------------------------------------------------------------------------------

Would I interpret this as 81% probability of a person not-under-treatment to have opioid poisoning and 98% for those under-treatment? So one is more likely to have an opioid poisoning if under-treatment? Those are large probabiliies but not statistically significantly different from each other?

Also, you mention that the i.month fixed effects are obviously very important here. So should I be adding the atmeans option for the margins command to capture those effects as well?

Thank you so much prof. Schechter!
Sincerely,
Sumedha.

[Clyde Schechter]

Would I interpret this as 81% probability of a person not-under-treatment to have opioid poisoning and 98% for those under-treatment? So one is more likely to have an opioid poisoning if under-treatment? Those are large probabiliies but not statistically significantly different from each other?

The outcome being estimated here is pu0, which is the probability of a positive outcome (opioid poisoning) conditional on the person-level fixed effect being 0. It is not possible to estimate the unconditional probability of a positive outcome in a fixed-effects logistic model. As for whether the difference between them is statistically significant, you cannot directly assess that from that margins output. In the -xtlogit- model, the coefficient of -did- is associated with a pvalue of 0.000, so certainly in the log-odds metric the effect is statistically significant. If you want to test it in the probability metric (again, conditional on fixed effect = 0), then re-run the same -margins- command adding the -pwcompare- option.

[Sumedha Gupta]

Dear Prof. Schechter,

Thank you for your invaluable advice and for continuing to repeatedly bothering you. I have another question resulting from re-specifying my treatment variable. I want to estimate the effect of the "duration" in treatment on the outcome of poisoning instead of just having received treatment. In economics we refer to it as an 'event-study' framework. But it is a straightforward manipulation where the month in which patient starts treatment is considered as the time origin=0. Then months prior to treatment get negative values (eg -1 implies 1 month prior to starting treatment, whereas 1 is 1 month after treatment starts). The idea simply being that the impact of the treatment can vary over time. I have recast my data to fit this framework and here is a dataex example for it:

* Example generated by -dataex-. To install: ssc install dataex
clear
input double studypersonid float(month monthtreat treat monthsTREAT)
103334  0  . 0   0
103334  1  . 0   0
103334  2  . 0   0
103334  3  . 0   0
103334  4  . 0   0
103334  5  . 0   0
103334  6  . 0   0
103334  7  . 0   0
103334  8  . 0   0
103334  9  . 0   0
103334 10  . 0   0
103334 11  . 0   0
103334 12  . 0   0
103334 13  . 0   0
103334 14  . 0   0
103334 15  . 0   0
103334 16  . 0   0
103334 17  . 0   0
103334 18  . 0   0
103334 19  . 0   0
103334 20  . 0   0
103334 21  . 0   0
103334 22  . 0   0
103334 23  . 0   0
103334 24  . 0   0
103334 25  . 0   0
103334 26  . 0   0
103334 27  . 0   0
103334 28  . 0   0
103334 29  . 0   0
103334 30  . 0   0
103334 31  . 0   0
103334 32  . 0   0
103334 33  . 0   0
103334 34  . 0   0
103334 35  . 0   0
103334 36  . 0   0
103334 37  . 0   0
103334 38  . 0   0
103334 39  . 0   0
103334 40  . 0   0
103334 41  . 0   0
103334 42  . 0   0
103334 43  . 0   0
103334 44  . 0   0
103334 45  . 0   0
103334 46  . 0   0
103334 47  . 0   0
103334 48  . 0   0
103334 49  . 0   0
103334 50  . 0   0
103334 51  . 0   0
103334 52  . 0   0
103334 53  . 0   0
103334 54  . 0   0
103334 55  . 0   0
103334 56  . 0   0
103334 57  . 0   0
103334 58  . 0   0
103334 59  . 0   0
103334 60  . 0   0
103334 61  . 0   0
103334 62  . 0   0
103335  0 48 1 -48
103335  1 48 1 -47
103335  2 48 1 -46
103335  3 48 1 -45
103335  4 48 1 -44
103335  5 48 1 -43
103335  6 48 1 -42
103335  7 48 1 -41
103335  8 48 1 -40
103335  9 48 1 -39
103335 10 48 1 -38
103335 11 48 1 -37
103335 12 48 1 -36
103335 13 48 1 -35
103335 14 48 1 -34
103335 15 48 1 -33
103335 16 48 1 -32
103335 17 48 1 -31
103335 18 48 1 -30
103335 19 48 1 -29
103335 20 48 1 -28
103335 21 48 1 -27
103335 22 48 1 -26
103335 23 48 1 -25
103335 24 48 1 -24
103335 25 48 1 -23
103335 26 48 1 -22
103335 27 48 1 -21
103335 28 48 1 -20
103335 29 48 1 -19
103335 30 48 1 -18
103335 31 48 1 -17
103335 32 48 1 -16
103335 33 48 1 -15
103335 34 48 1 -14
103335 35 48 1 -13
103335 36 48 1 -12
end

------------------ copy up to and including the previous line ------------------

Listed 100 out of 182223 observations
Use the count() option to list more

Individual 1 is in the control group and individual 2 is in the treated group and received treatment in month 48. Due to dataex 100 observation limit the example stops at monthsTREAT=-12 for the second person.

To estimate the time varying impact of the treatment I created dummies for each value of monthsTREAT by:

And, then I estimated:

xtlogit anypois i.month i.point t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 t13 t14 t15 t16 t17 t18 t19 t20 t21 t22 t23 t24 t25 t26 t27 t28 ///
t29 t30 t31 t32 t33 t34 t35 t36 t37 t38 t39 t40 t41 t42 t43 t44 t45 t46 t47 t48 t50 t51 t52 t53 t54 t55 t56 t57 t58 t59 t60 ///
t61 t62 t63 t64 t65 t66 t67 t68 t69 t70 t71 t72 t73 t74, fe

note: multiple positive outcomes within groups encountered.
note: 163 groups (9,740 obs) dropped because of all positive or
all negative outcomes.
note: 1.point omitted because of no within-group variance.

Iteration 0: log likelihood = -14742.026
Iteration 1: log likelihood = -14727.814
Iteration 2: log likelihood = -14723.745
Iteration 3: log likelihood = -14723.16
Iteration 4: log likelihood = -14723.021
Iteration 5: log likelihood = -14722.991
Iteration 6: log likelihood = -14722.984
Iteration 7: log likelihood = -14722.983
Iteration 8: log likelihood = -14722.983
Iteration 9: log likelihood = -14722.983

Conditional fixed-effects logistic regression Number of obs = 172,483
Group variable: studypersonid Number of groups = 2,758

Obs per group:
min = 37
avg = 62.5
max = 63

LR chi2(135) = 5179.51
Log likelihood = -14722.983 Prob > chi2 = 0.0000

------------------------------------------------------------------------------
anypois | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
month |
1 | .2880636 .7642568 0.38 0.706 -1.209852 1.785979
2 | .2880635 .7642568 0.38 0.706 -1.209852 1.785979
3 | .4699059 .7321908 0.64 0.521 -.9651617 1.904974
4 | -.4212534 .9134515 -0.46 0.645 -2.211585 1.369079
5 | .626903 .7095707 0.88 0.377 -.76383 2.017636
6 | .6357031 .709353 0.90 0.370 -.7546032 2.026009
7 | -.4366951 .9137868 -0.48 0.633 -2.227684 1.354294
8 | -.4636114 .9148315 -0.51 0.612 -2.256648 1.329425
9 | .794591 .6918254 1.15 0.251 -.5613619 2.150544
10 | .2645759 .7644926 0.35 0.729 -1.233802 1.762954
11 | 1.15898 .6596685 1.76 0.079 -.1339466 2.451906
12 | -.0456184 .8179271 -0.06 0.956 -1.648726 1.557489
13 | 1.084194 .6673592 1.62 0.104 -.223806 2.392194
14 | 1.052991 .6678773 1.58 0.115 -.2560246 2.362006
15 | 1.074929 .6675593 1.61 0.107 -.2334634 2.383321
16 | .6528311 .7082228 0.92 0.357 -.7352601 2.040922
17 | 1.248584 .6526731 1.91 0.056 -.0306323 2.527799
18 | .935773 .6780533 1.38 0.168 -.393187 2.264733
19 | 1.039859 .6679874 1.56 0.120 -.2693721 2.349091
20 | 1.608035 .6304883 2.55 0.011 .3723007 2.843769
21 | 1.151603 .6593892 1.75 0.081 -.1407766 2.443982
22 | 1.549423 .6336011 2.45 0.014 .3075875 2.791258
23 | 1.414241 .6416167 2.20 0.028 .1566954 2.671787
24 | 1.046392 .6676284 1.57 0.117 -.2621356 2.35492
25 | 1.421983 .6413674 2.22 0.027 .164926 2.67904
26 | 2.004475 .6146624 3.26 0.001 .7997591 3.209191
27 | 2.14995 .6094843 3.53 0.000 .9553828 3.344517
28 | 2.720422 .5962843 4.56 0.000 1.551727 3.889118
29 | 2.459005 .6016633 4.09 0.000 1.279766 3.638243
30 | 2.65211 .597515 4.44 0.000 1.481002 3.823218
31 | 2.431615 .6023317 4.04 0.000 1.251066 3.612163
32 | 2.556066 .599804 4.26 0.000 1.380472 3.731661
33 | 3.345532 .5881468 5.69 0.000 2.192785 4.498279
34 | 3.794456 .5846461 6.49 0.000 2.648571 4.940341
35 | 3.798109 .5846249 6.50 0.000 2.652265 4.943952
36 | 3.693564 .585262 6.31 0.000 2.546471 4.840656
37 | 3.595188 .5859579 6.14 0.000 2.446732 4.743644
38 | 3.795807 .5845964 6.49 0.000 2.650019 4.941595
39 | 3.822806 .584396 6.54 0.000 2.67741 4.968201
40 | 3.927917 .5838414 6.73 0.000 2.783609 5.072225
41 | 4.196958 .5825636 7.20 0.000 3.055154 5.338762
42 | 3.969196 .583646 6.80 0.000 2.825271 5.113121
43 | 4.204605 .5825626 7.22 0.000 3.062803 5.346406
44 | 4.268401 .5823127 7.33 0.000 3.127089 5.409713
45 | 4.192959 .5826247 7.20 0.000 3.051036 5.334883
46 | 4.289015 .5822465 7.37 0.000 3.147833 5.430197
47 | 4.173114 .582756 7.16 0.000 3.030933 5.315295
48 | 4.352529 .5820532 7.48 0.000 3.211725 5.493332
49 | 4.34676 .5821106 7.47 0.000 3.205844 5.487676
50 | 3.902377 .5842623 6.68 0.000 2.757244 5.04751
51 | 4.060147 .5834112 6.96 0.000 2.916682 5.203612
52 | 4.116284 .5831557 7.06 0.000 2.97332 5.259249
53 | 4.320525 .5822882 7.42 0.000 3.179262 5.461789
54 | 4.161532 .5829798 7.14 0.000 3.018912 5.304151
55 | 4.168578 .5829798 7.15 0.000 3.025958 5.311197
56 | 3.848953 .5847516 6.58 0.000 2.702861 4.995045
57 | 3.181196 .5908103 5.38 0.000 2.023229 4.339163
58 | 3.134223 .5914174 5.30 0.000 1.975066 4.29338
59 | 3.072511 .5922948 5.19 0.000 1.911635 4.233388
60 | 2.795383 .5966302 4.69 0.000 1.626009 3.964757
61 | 3.087489 .5921126 5.21 0.000 1.92697 4.248008
62 | 2.494943 .603038 4.14 0.000 1.31301 3.676876
|
1.point | 0 (omitted)
t1 | -15.9049 28441.26 -0.00 1.000 -55759.76 55727.95
t2 | -15.90249 20433.29 -0.00 0.999 -40064.42 40032.61
t3 | -15.90668 13660.17 -0.00 0.999 -26789.35 26757.53
t4 | -15.90311 10628.43 -0.00 0.999 -20847.24 20815.44
t5 | -15.905 10777.25 -0.00 0.999 -21138.92 21107.11
t6 | -15.92968 8294.378 -0.00 0.998 -16272.61 16240.75
t7 | -15.94135 7585.618 -0.00 0.998 -14883.48 14851.6
t8 | -15.94621 7313.475 -0.00 0.998 -14350.09 14318.2
t9 | -15.96144 5651.532 -0.00 0.998 -11092.76 11060.84
t10 | -15.9699 5360.576 -0.00 0.998 -10522.51 10490.57
t11 | -15.98289 4998.281 -0.00 0.997 -9812.434 9780.468
t12 | -15.97901 4067.741 -0.00 0.997 -7988.606 7956.648
t13 | -15.97673 4418.353 -0.00 0.997 -8675.79 8643.837
t14 | -15.98038 4117.451 -0.00 0.997 -8086.037 8054.076
t15 | .5317691 1.093148 0.49 0.627 -1.610761 2.674299
t16 | -15.97405 4272.469 -0.00 0.997 -8389.859 8357.911
t17 | .5653975 1.095464 0.52 0.606 -1.581672 2.712467
t18 | -15.97594 3644.619 -0.00 0.997 -7159.298 7127.347
t19 | -15.979 3882.136 -0.00 0.997 -7624.825 7592.867
t20 | -15.98465 3372.791 -0.00 0.996 -6626.534 6594.565
t21 | .2394105 1.084247 0.22 0.825 -1.885674 2.364495
t22 | -15.98635 3300.501 -0.00 0.996 -6484.849 6452.876
t23 | -15.9908 2865.405 -0.01 0.996 -5632.082 5600.1
t24 | -15.97909 3031.959 -0.01 0.996 -5958.509 5926.55
t25 | -15.98894 2955.434 -0.01 0.996 -5808.534 5776.556
t26 | -.123772 1.087425 -0.11 0.909 -2.255086 2.007542
t27 | -15.98401 2772.716 -0.01 0.995 -5450.408 5418.44
t28 | -.105081 1.083005 -0.10 0.923 -2.227733 2.017571
t29 | -.4380744 1.081096 -0.41 0.685 -2.556984 1.680835
t30 | -.3975106 1.076747 -0.37 0.712 -2.507897 1.712875
t31 | -.5746848 1.080819 -0.53 0.595 -2.693051 1.543681
t32 | .0002732 .8167529 0.00 1.000 -1.600533 1.601079
t33 | -.6702825 1.082311 -0.62 0.536 -2.791573 1.451009
t34 | -.9177851 1.086735 -0.84 0.398 -3.047747 1.212177
t35 | -15.99042 1673.947 -0.01 0.992 -3296.866 3264.885
t36 | -.5759103 .8171163 -0.70 0.481 -2.177429 1.025608
t37 | -1.600506 1.080724 -1.48 0.139 -3.718687 .5176747
t38 | -1.70732 1.077663 -1.58 0.113 -3.819501 .4048599
t39 | -.3290586 .6429183 -0.51 0.609 -1.589155 .9310381
t40 | -.4650269 .6449098 -0.72 0.471 -1.729027 .7989731
t41 | -16.06639 1077.933 -0.01 0.988 -2128.776 2096.643
t42 | -1.517617 .815622 -1.86 0.063 -3.116207 .080973
t43 | -1.228602 .7043873 -1.74 0.081 -2.609176 .1519718
t44 | -2.575047 1.078982 -2.39 0.017 -4.689812 -.4602811
t45 | -2.691755 1.075191 -2.50 0.012 -4.79909 -.5844199
t46 | -2.028379 .8102112 -2.50 0.012 -3.616363 -.4403939
t47 | -2.10685 .8128832 -2.59 0.010 -3.700072 -.5136279
t48 | -.6739323 .5441462 -1.24 0.216 -1.740439 .3925746
t50 | -1.50441 .6383752 -2.36 0.018 -2.755603 -.253218
t51 | -.5144505 .5134708 -1.00 0.316 -1.520835 .4919337
t52 | -.5099002 .5150822 -0.99 0.322 -1.519443 .4996424
t53 | -1.530935 .641107 -2.39 0.017 -2.787481 -.274388
t54 | -1.836051 .7044017 -2.61 0.009 -3.216653 -.455449
t55 | -.430036 .5062716 -0.85 0.396 -1.42231 .562238
t56 | -1.882658 .7013691 -2.68 0.007 -3.257317 -.5080002
t57 | -3.057162 1.079405 -2.83 0.005 -5.172757 -.9415667
t58 | -3.069731 1.08069 -2.84 0.005 -5.187844 -.9516175
t59 | -1.776944 .7061637 -2.52 0.012 -3.161 -.3928889
t60 | -1.391809 .6462915 -2.15 0.031 -2.658517 -.1251009
t61 | -2.137131 .8211609 -2.60 0.009 -3.746577 -.5276855
t62 | -2.837792 1.083813 -2.62 0.009 -4.962026 -.7135572
t63 | -2.77225 1.082141 -2.56 0.010 -4.893207 -.651293
t64 | -1.527893 .7107902 -2.15 0.032 -2.921016 -.1347693
t65 | -16.20198 862.8628 -0.02 0.985 -1707.382 1674.978
t66 | -2.312075 1.08622 -2.13 0.033 -4.441026 -.1831239
t67 | -16.18374 1092.471 -0.01 0.988 -2157.388 2125.02
t68 | -1.888274 1.090744 -1.73 0.083 -4.026093 .2495459
t69 | -16.11064 1467.32 -0.01 0.991 -2892.006 2859.784
t70 | -1.357767 1.099858 -1.23 0.217 -3.513449 .7979159
t71 | -16.1167 1722.673 -0.01 0.993 -3392.494 3360.261
t72 | -.4948469 1.115573 -0.44 0.657 -2.681329 1.691635
t73 | -16.17779 2912.311 -0.01 0.996 -5724.203 5691.848
t74 | -16.18091 4444.487 -0.00 0.997 -8727.216 8694.854
------------------------------------------------------------------------------

.
.
end of do-file



So I see from the coefficients on the t* (months since treatment dummies) that timing is important. I wish to run the margins command to estimate the marginal effect for each t* variable but it will not run.

margins t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 t13 t14 t15 t16 t17 t18 t19 t20 t21 t22
> t23 t24 t25 t26 t27 t28 ///
> t29 t30 t31 t32 t33 t34 t35 t36 t37 t38 t39 t40 t41 t42 t43 t44 t45 t46 t47 t48 t50 t51
> t52 t53 t54 t55 t56 t57 t58 t59 t60 ///
> t61 t62 t63 t64 t65 t66 t67 t68 t69 t70 t71 t72 t73 t74, pwcompare
factor 't1' not found in list of covariates
r(322);


These are binaries so it should run? I will be very grateful for your advise. Thank you again Prof. Schechter.

Sincerely,
Sumedha.

[Clyde Schechter]

These are binaries so it should run? I will be very grateful for your advise.

They may indeed be binaries, but you didn't specify them that way in your -xtlogit- command, so -margins- doesn't know that they are. -help fvvarlist-. The i. prefix is not optional.

I also suggest that before proceeding with this you consult your supervisor/advisor about the appropriateness of this model. What you are doing here is "legal" but it is hard for me to imagine that anyone can make any sense out of 74 distinct time effects. Looking at the coefficients of those variables "from 50,000 feet up", it seem that you have more or less of a trend going from very strongly negative to slightly positive as you run through the series, peaking around t32, and then declining again to large negatives. The sequence is not strictly monotone on either side of t32, but the overall appearance of a trend is quite striking. Moreover, this is precisely what I would expect: the farther away in time you are from the "event" the smaller the effect. I would imagine that this could be better handled by using a continuous representation of monthsTreat rather than a mob of dummies. Simply including it as a single continuous variable would not work out, as that would imply a linear relationship, that is clearly not what you have. But a linear or cubic spline could work quite nicely. As I do not work in your area, I don't know what would be understood and considered persuasive in your discipline in this regard, but your supervisor or advisor presumably would.