Non Linear Difference in difference

[Neeraj Kumar]

Hi,
I am using difference in difference method with the help of logit model. I have read this paper " Interaction term in logit and probit models" by Churnog Ai and Edward C. Nortan. In that paper they described that interaction effect requires computing Cross derivative and Cross difference. I have gone through this paper as well " The treatment effect, the cross difference, and the interaction term in nonlinear" by Patrick A. Phuani (2012). In which he described that in the case of non Linear difference in difference the treatment effect i.e the parameter of interest, is not a simple cross difference , but a difference between cross differences: it is the cross difference of the conditional expectation of the observed outcome minus the cross difference of the conditional expectation of the potential outcome without treatment and this difference in cross differences simplifies to the incremental effect of the coefficient of the interaction term. so that treatment effect has the same sign as the coefficient of the interaction effect.
I am using a two year paneI data (2004-05=0 and 2011-12 = 1). time variable is a discrete variable having value 1 and 0. mgnregadmy is dummy variable (treatment group = 1 and control group= 0) used the following command to compute DID in the case of Logit Model.

logit  Loan20 i.mgnregadmy##i. time  RO5 ca2 ca3 education1 NPERSONS COPC, vce(robust)

margins mgnregadmy, dydx (time) pwcompare

I have a doubt that, Is the treatment effect is measured by the interaction variable (mgnregadmy*time) or is it measured by the pairwise comparison of average marginal effect? if it is measured by the pairwise comparison of average marginal effect, then how can it be difference in difference estimator because DID includes the double differences. In this case is pairwise comparison of average marginal effect show the cross difference between two cross differences?

[Dimitriy V. Masterov]

Here's an example where I model the probability of a restaurant having more than 20 employees after a minimum wage law is implemented using a logit DID and an OLS specification. Here NJ restaurants are the treated group and there are two periods. Using a replicable example on a standard dataset is good practice.
Also, if you have heteroskedasticity in a logit, you coefficients are not identified, and adding robust won't help.

It shows that your code is calculating the difference the differences and that it matches the OLS parameter (this happens because the logit specification contains only fully-interacted dummies).

Your way of writing the contrast of margins is not the clearest to me, but it is correct. You can see that the DID effect is 100*[(.3625378-.2688822)-(.3417722-.3417722)]=9.4%, though not significant. The PA prediction is very similar in both years since the effect of time is very small, which makes the predictions look a tad strange.

. use http://fmwww.bc.edu/repec/bocode/c/CardKrueger1994.dta, clear
(Dataset from Card&Krueger (1994))

. gen fte_gt_20 = cond(fte>20,1,0)

. /* Logit DID */
. qui logit fte_gt_20 i.treated##i.t, nolog

. margins treated#t, nopvalues

Adjusted predictions                            Number of obs     =        820
Model VCE    : OIM

Expression   : Pr(fte_gt_20), predict()

--------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.     [95% Conf. Interval]
-------------+------------------------------------------------
   treated#t |
       PA#0  |   .3417722   .0533633      .2371819    .4463624
       PA#1  |   .3417722   .0533633      .2371819    .4463624
       NJ#0  |   .2688822   .0243703      .2211173    .3166471
       NJ#1  |   .3625378   .0264235      .3107487    .4143268
--------------------------------------------------------------

. margins r.treated#r.t

Contrasts of adjusted predictions
Model VCE    : OIM

Expression   : Pr(fte_gt_20), predict()

------------------------------------------------
             |         df        chi2     P>chi2
-------------+----------------------------------
   treated#t |          1        1.26     0.2625
------------------------------------------------

----------------------------------------------------------------------
                     |            Delta-method
                     |   Contrast   Std. Err.     [95% Conf. Interval]
---------------------+------------------------------------------------
           treated#t |
(NJ vs PA) (1 vs 0)  |   .0936556   .0835907     -.0701791    .2574903
----------------------------------------------------------------------

. margins treated, dydx(t) pwcompare // same as above, but less clear

Pairwise comparisons of conditional marginal effects
Model VCE    : OIM

Expression   : Pr(fte_gt_20), predict()
dy/dx w.r.t. : 1.t

--------------------------------------------------------------
             |   Contrast Delta-method         Unadjusted
             |      dy/dx   Std. Err.     [95% Conf. Interval]
-------------+------------------------------------------------
1.t          |
     treated |
   NJ vs PA  |   .0936556   .0835907     -.0701791    .2574903
--------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the
      base level.

. /* OLS DID */
. reg fte_gt_20 i.treated##i.t, robust

Linear regression                               Number of obs     =        820
                                                F(3, 816)         =       2.42
                                                Prob > F          =     0.0648
                                                R-squared         =     0.0086
                                                Root MSE          =     .46588

------------------------------------------------------------------------------
             |               Robust
   fte_gt_20 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     treated |
         NJ  |    -.07289   .0588084    -1.24   0.216    -.1883235    .0425436
         1.t |   5.43e-16   .0756519     0.00   1.000    -.1484952    .1484952
             |
   treated#t |
       NJ#1  |   .0936556   .0837953     1.12   0.264    -.0708241    .2581353
             |
       _cons |   .3417722    .053494     6.39   0.000     .2367702    .4467741
------------------------------------------------------------------------------

[Neeraj Kumar]

Tnx for reply Mr. Masterov. So it's mean treatment effect is measure by the pairwise comparison of conditional margins. So this dy/dx value is cross difference of two cross differences (according to phuani) ( Correct me if I m wrong). What will be the interpretation of this DID estimator(dy/dx). What is the role of coefficient of interaction variable in this case if the effect of program estimated by dy/dx.

[Dimitriy V. Masterov]

I think I misunderstood what you were asking. I thought you wanted to know if you were estimating the cross-difference, to which the answer is yes. Here is code also showing the correct Puhani DID estimator. Note that I tweaked the outcome slightly.

use http://fmwww.bc.edu/repec/bocode/c/CardKrueger1994.dta, clear
gen fte_gt_15 = cond(fte>15,1,0)
/* Logit Cross-Difference DID (equation 14) */
logit fte_gt_15 i.treated##i.t bk kfc roys , nolog
margins treated#t, nopvalues
margins r.treated#r.t
margins treated, dydx(t) pwcompare // same as above, but less clear

gen tg = treated*t
logit fte_gt_15 i.(treated t tg) bk kfc roys, nolog
/* Puhani's DID Estimator (equation 10) */
margins, at(treated==1 t==1 tg ==1) at(treated==1 t==1 tg ==0) contrast(atcontrast(a._at) wald)

[Neeraj Kumar]

Thank you so much Mr. Masterov. I used the above commands which are mention #4. Since in Puhani's paper mention that treatment effect is shown by equation 10. so for my analysis i used the commands, which is suitable for equation 10. i just want to be sure whether the interpretation i did is correct or not, specially in the case of mgnregadmy (treatment), time and contrast of predictive margin (correct me if i did wrong interpretation of any of the three variables.). Information regarding variables are given below:
I used the following commands

logit  Loan20 i.(mgnregadmy time  intereaction)  RO5 ca2 ca3 education1 NPERSONS COPC, nolog

and

margins, at(mgnregadmy==1 time==1 intereaction==1) at (mgnregadmy==1 time==1 intereaction==0) contrast (atcontrast (a._at) wald)

. Here my dependent variable is Loan20. this having value 1 for formal sources and 0 for informal sources. mgnregadmy shows 1 for treatment group and 0 for control group. and time is 1 for year 2011-12 and 0 for 2004-05.
Results are as follows:

. logit  Loan20 i.(mgnregadmy time  intereaction)  RO5 ca2 ca3 education1 NPERSONS COPC, nolog

Logistic regression                               Number of obs   =      24562
                                                  LR chi2(9)      =    2443.86
                                                  Prob > chi2     =     0.0000
Log likelihood =   -15038.2                       Pseudo R2       =     0.0751

------------------------------------------------------------------------------
      Loan20 |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.mgnregadmy |  -.3960967   .0477788    -8.29   0.000    -.4897415    -.302452
      1.time |   .3499736   .0401513     8.72   0.000     .2712785    .4286687
1.intereac~n |   .1660455   .0587435     2.83   0.005     .0509105    .2811806
         RO5 |   .0229627   .0011288    20.34   0.000     .0207503    .0251751
         ca2 |  -.3157375   .0349427    -9.04   0.000     -.384224    -.247251
         ca3 |  -.3948137   .0395491    -9.98   0.000    -.4723286   -.3172988
  education1 |   .7707443   .0302849    25.45   0.000     .7113868    .8301017
    NPERSONS |   .0501983   .0056809     8.84   0.000      .039064    .0613326
        COPC |    .000117   .0000101    11.57   0.000     .0000972    .0001369
       _cons |  -2.367033   .0795032   -29.77   0.000    -2.522857    -2.21121
------------------------------------------------------------------------------

and

. margins, at(mgnregadmy==1 time==1 intereaction==1) at (mgnregadmy==1 time==1 intereaction==0)
>  contrast (atcontrast (a._at) wald)

Contrasts of predictive margins
Model VCE    : OIM

Expression   : Pr(Loan20), predict()

1._at        : mgnregadmy      =           1
               time            =           1
               intereaction    =           1

2._at        : mgnregadmy      =           1
               time            =           1
               intereaction    =           0

------------------------------------------------
             |         df        chi2     P>chi2
-------------+----------------------------------
         _at |          1        8.20     0.0042
------------------------------------------------

--------------------------------------------------------------
             |            Delta-method
             |   Contrast   Std. Err.     [95% Conf. Interval]
-------------+------------------------------------------------
         _at |
   (1 vs 2)  |   .0356693    .012455       .011258    .0600805
--------------------------------------------------------------

.

The probability of expected mean difference of having formal loan in comparison to informal loan between treatment group and control group are -0.396 percent unit. This is showed by the coefficient of mgregadmy. The time coefficient is 0.349. This shows that there is increase in probability of getting formal loan in comparison to informal loan for those who are not participating in program. The differences between two cross differences is 0.028 percent unit. This shows that the probability of taking formal loan in comparison to informal loan for treatment group has 0.0356 percent unit more. Correct me if this interpretation is wrong.

[Dimitriy V. Masterov]

This sounds good to me.

[Neeraj Kumar]

Thank you Mr. Masterov.

[Ke Ju]

Dear Mr. Masterov and Mr. Kumar,

Thank you very much for the discussion, one question I have is if my dependent variable is an ordered categorical variable, is there a good solution?
For example, take the following code as an example:

code:

. use http://fmwww.bc.edu/repec/bocode/c/CardKrueger1994.dta, clear
. gen fte_gt = 0
. replace fte_gt = 1 if fte >= 15 & fte <= 20
. replace fte_gt = 2 if fte > 20

. ologit fte_gt i.treated##i.t

Iteration 0:   log likelihood = -891.35925  
Iteration 1:   log likelihood = -888.60219  
Iteration 2:   log likelihood = -888.60179  
Iteration 3:   log likelihood = -888.60179  

Ordered logistic regression                     Number of obs     =        820
                                                LR chi2(3)        =       5.51
                                                Prob > chi2       =     0.1377
Log likelihood = -888.60179                     Pseudo R2         =     0.0031

------------------------------------------------------------------------------
      fte_gt |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     treated |
         NJ  |  -.3304535    .229076    -1.44   0.149    -.7794343    .1185273
         1.t |  -.0275562   .2911727    -0.09   0.925    -.5982442    .5431318
             |
   treated#t |
       NJ#1  |   .3312446   .3249211     1.02   0.308     -.305589    .9680782
-------------+----------------------------------------------------------------
       /cut1 |  -.5515638   .2081406                     -.9595118   -.1436158
       /cut2 |   .6056132   .2083374                      .1972795    1.013947
------------------------------------------------------------------------------


. margins treated#t

Adjusted predictions                            Number of obs     =        820
Model VCE    : OIM

1._predict   : Pr(fte_gt==0), predict(pr outcome(0))
2._predict   : Pr(fte_gt==1), predict(pr outcome(1))
3._predict   : Pr(fte_gt==2), predict(pr outcome(2))

------------------------------------------------------------------------------------
                   |            Delta-method
                   |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------------+----------------------------------------------------------------
_predict#treated#t |
           1#PA#0  |   .3655017   .0482699     7.57   0.000     .2708944     .460109
           1#PA#1  |   .3719156    .048834     7.62   0.000     .2762027    .4676285
           1#NJ#0  |   .4449465   .0258516    17.21   0.000     .3942784    .4956147
           1#NJ#1  |   .3717308   .0251531    14.78   0.000     .3224316    .4210301
           2#PA#0  |   .2814378   .0158507    17.76   0.000      .250371    .3125046
           2#PA#1  |   .2812922   .0158883    17.70   0.000     .2501517    .3124326
           2#NJ#0  |   .2733579   .0157735    17.33   0.000     .2424425    .3042734
           2#NJ#1  |   .2812977   .0158378    17.76   0.000     .2502563    .3123392
           3#PA#0  |   .3530605   .0475861     7.42   0.000     .2597935    .4463275
           3#PA#1  |   .3467922   .0474462     7.31   0.000     .2537993    .4397851
           3#NJ#0  |   .2816955   .0221846    12.70   0.000     .2382146    .3251765
           3#NJ#1  |   .3469714    .024574    14.12   0.000     .2988073    .3951356
------------------------------------------------------------------------------------


. margins r.treated#r.t


Contrasts of adjusted predictions
Model VCE    : OIM

1._predict   : Pr(fte_gt==0), predict(pr outcome(0))
2._predict   : Pr(fte_gt==1), predict(pr outcome(1))
3._predict   : Pr(fte_gt==2), predict(pr outcome(2))

----------------------------------------------------------
                       |         df        chi2     P>chi2
-----------------------+----------------------------------
    treated#t@_predict |
(NJ vs PA) (1 vs 0) 1  |          1        1.09     0.2954
(NJ vs PA) (1 vs 0) 2  |          1        2.72     0.0994
(NJ vs PA) (1 vs 0) 3  |          1        0.96     0.3278
                Joint  |          2        2.72     0.2573
----------------------------------------------------------

------------------------------------------------------------------------
                       |            Delta-method
                       |   Contrast   Std. Err.     [95% Conf. Interval]
-----------------------+------------------------------------------------
    treated#t@_predict |
(NJ vs PA) (1 vs 0) 1  |  -.0796296   .0761059     -.2287944    .0695351
(NJ vs PA) (1 vs 0) 2  |   .0080854    .004907     -.0015321     .017703
(NJ vs PA) (1 vs 0) 3  |   .0715442   .0731111     -.0717509    .2148394
------------------------------------------------------------------------


. gen tg = treated*t
. ologit fte_gt i.(treated t tg) bk kfc roys

Iteration 0:   log likelihood = -891.35925  
Iteration 1:   log likelihood = -783.15648  
Iteration 2:   log likelihood = -782.16628  
Iteration 3:   log likelihood = -782.16161  
Iteration 4:   log likelihood =  -782.1616  

Ordered logistic regression                     Number of obs     =        820
                                                LR chi2(6)        =     218.40
                                                Prob > chi2       =     0.0000
Log likelihood =  -782.1616                     Pseudo R2         =     0.1225

------------------------------------------------------------------------------
      fte_gt |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     treated |
         NJ  |  -.2485957   .2416733    -1.03   0.304    -.7222667    .2250753
         1.t |  -.0341401   .3056148    -0.11   0.911    -.6331341    .5648538
        1.tg |   .4041914    .342902     1.18   0.239    -.2678842    1.076267
          bk |    .221209   .1965724     1.13   0.260    -.1640658    .6064838
         kfc |   -2.91014   .3010229    -9.67   0.000    -3.500134   -2.320146
        roys |  -.2896749    .214392    -1.35   0.177    -.7098755    .1305257
-------------+----------------------------------------------------------------
       /cut1 |  -.9599848   .2655394                     -1.480432   -.4395372
       /cut2 |   .4281474   .2631292                     -.0875763    .9438711
------------------------------------------------------------------------------

[Yadav Sandeep]

Dear Mr. Masterov, Kumar and Ju,

I am using Probit DID and I want to estimate the effect of interction term (binary variable and categorical variable) which command should I use? Commands discussed above are not working for me.