Non stopping iterations problem with teffects ipwra

[Merve Kucuk]

Hi all,

I am trying for the first time to run my difference-in-difference model with inverse propensity score matching. I have an unbalanced longitudinal dataset. So far I was running the below code:

xtreg logcleanhwtrusi FY2007 FY2008 FY2009 FY2011 FY2012 FY2013 didFY2007 didFY2008 didFY2009 didFY2011 didFY2012 didFY2013, fe

where

logcleanhwtrusi

is (one of) my outcome variable,

FY2007 FY2008 FY2009 FY2011 FY2012 FY2013

are my year fixed effects

didFY2008 didFY2009 didFY2011 didFY2012 didFY2013

are my treatment effects
and I control for fixed effects.

This way I was getting a result. But now I wanted to use "Regression adjustment with IPW".

So I ran the below code

teffects ipwra (logcleanhwtrusi FY2007 FY2008 FY2009 FY2011 FY2012 FY2013 didFY2007 didFY2008 didFY2009 didFY2011 didFY2012 didFY2013) (intreatmentgroup cleanedhigh1 cleanesempst hgsex hhtype cleanhstenr)

But due to some reason I cannot get the final result and keep getting line by line all the iterations which do not seem to end my output is as follows:

note: instrument __000003 omitted because of collinearity
Iteration 0:   EE criterion =  3.003e-17  (not concave)
Iteration 1:   EE criterion =  9.017e-18  (not concave)
Iteration 2:   EE criterion =  1.458e-18  (not concave)
Iteration 3:   EE criterion =  6.304e-19  (not concave)
Iteration 4:   EE criterion =  3.107e-19  (not concave)
Iteration 5:   EE criterion =  2.119e-19  (not concave)
Iteration 6:   EE criterion =  1.572e-19  (not concave)
Iteration 7:   EE criterion =  1.152e-19  (not concave)
Iteration 8:   EE criterion =  9.427e-20  (not concave)
Iteration 9:   EE criterion =  7.718e-20  (not concave)
Iteration 10:  EE criterion =  6.520e-20  (not concave)
Iteration 11:  EE criterion =  5.569e-20  (not concave)
Iteration 12:  EE criterion =  4.762e-20  (not concave)

I have waited for it to come to an end but it doesn't end (in a normal time). I feel like I have been doing a logical mistake somewhere. Is there anyone out there who could help me with this? I have read through the stata document about this function and watched some videos but I cannot figure it out. Any help would be highly appreciated. Thank you very very much in advance.

[FernandoRios]

Hi Merve,
First things first. If you see a problem like this while using teffects, the usual culprit is the first step (either probit or logit)
Perhaps you can start and seeing if the first stage converges:

probit intreatmentgroup cleanedhigh1 cleanesempst hgsex hhtype cleanhstenr

and/or explore when is it that it stops converging.
Once that is "solved", my guess is that the teffects will run without further problems.
HTH
Fernando

[Merve Kucuk]

Hi Fernando,

Thank you very much for your answer. I have tried it and it converges without any problems in 3 iterations. Does this mean that I am making something else wrong?
Thank you very much for your help.

Best,
Merve

[FernandoRios]

That was unexpected. Ok, so my next suggestion is trying to replicate what teffect ipwra does by hand, and see in which step it gives you a problem.
for this:
1. Run probit (done)
2. Obtain the predicted probabilities (predict pr)
3. Define inverse probability weights for,,,say,,, ATE (gen ipw=(intreatment==1)/pr+(intreatment==0)/(1-pr))
4. Run the following regressions:

1. this is kind of a 1 step IPWRA. just to check if everything works 
 reg logcleanhwtrusi intreatmentgroup FY2007 FY2008 FY2009 FY2011 FY2012 FY2013 ///  didFY2007 didFY2008 didFY2009 didFY2011 didFY2012 didFY2013 [pw=ipw]  2. Run separate regressions:  
 reg logcleanhwtrusi FY2007 FY2008 FY2009 FY2011 FY2012 FY2013 /// didFY2007 didFY2008 didFY2009 didFY2011 didFY2012 didFY2013 [pw=ipw] if intreatmentgroup ==1  
 reg logcleanhwtrusi FY2007 FY2008 FY2009 FY2011 FY2012 FY2013 /// didFY2007 didFY2008 didFY2009 didFY2011 didFY2012 didFY2013 [pw=ipw] if intreatmentgroup ==0

The problem may be coming from this step. Perhaps some coefficients are dropped, or have very large standard errors.

HTH
Fernando
PS. you are doing nothing wrong. what happens is that, i believe, teffects estimates the three equation model using GMM, and at some steps of the maximization, is simply having problems finding the solution.

[Merve Kucuk]

Hi Fernando,

Thank you for your answer. I tried to run all three codes but they did not work unfortunately. I got the error message

ipw not found

. Do I need to define it first? If so what is it? Sorry for bothering you over and over again. Thank you very much.

Kind regards
Merve

[FernandoRios]

Hi Merve,
You need to create ipw. Following the steps 1,2 and 3.

probit intreatmentgroup cleanedhigh1 cleanesempst hgsex hhtype cleanhstenr
predict pr
gen ipw=1/pr*(intreatmentgroup==1) + 1/(1-pr)*(intreatmentgroup==0)

then run the joint and separate regressions.
Fernando

[Merve Kucuk]

Hi Fernando,

I followed all these steps and the regressions all run smoothly.. Can there be another reason why the main regression doesn't work? Thank you so much for helping me..

Kind regards
Merve

[FernandoRios]

Can you share the outcomes of all regressions?

[Merve Kucuk]

Hi Fernando,

Of course. I wanted to provide you with a snapshot of the data.

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(logcleanhwtrusi FY2007 FY2008 FY2009 FY2011 FY2012 FY2013 didFY2007 didFY2008 didFY2009 didFY2011 didFY2012 didFY2013)
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 1 0 0 0 0 0 0 0 0 0 0 0
        . 0 1 0 0 0 0 0 0 0 0 0 0
        . 0 0 1 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 1 0 0 0 0 0 0 0 0
        . 0 0 0 0 1 0 0 0 0 0 0 0
14.508657 0 0 0 0 0 1 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 1 0 0 0 0 0 0 0 0 0 0 0
        . 0 1 0 0 0 0 0 0 0 0 0 0
        . 0 0 1 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 1 0 0 0 0 0 0 0 0
        . 0 0 0 0 1 0 0 0 0 0 0 0
        . 0 0 0 0 0 1 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 1 0 0 0 0 0 0 0 0 0 0 0
        . 0 1 0 0 0 0 0 0 0 0 0 0
        . 0 0 1 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 1 0 0 0 0 0 0 0 0
        . 0 0 0 0 1 0 0 0 0 0 0 0
        . 0 0 0 0 0 1 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 1 0 0 0 0 0 0 0 0 0 0 0
        . 0 1 0 0 0 0 0 0 0 0 0 0
        . 0 0 1 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 1 0 0 0 0 0 0 0 0
        . 0 0 0 0 1 0 0 0 0 0 0 0
        . 0 0 0 0 0 1 0 0 0 0 0 0
        . . . . . . . . . . . . .
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 1 0 0 0 0 0 0 0 0 0 0 0
        . 0 1 0 0 0 0 0 0 0 0 0 0
        . 0 0 1 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 1 0 0 0 0 0 0 0 0
        . 0 0 0 0 1 0 0 0 0 0 0 0
        . 0 0 0 0 0 1 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 1 0 0 0 0 0 0 0 0 0 0 0
        . 0 1 0 0 0 0 0 0 0 0 0 0
        . 0 0 1 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 1 0 0 0 0 0 0 0 0
        . 0 0 0 0 1 0 0 0 0 0 0 0
        . 0 0 0 0 0 1 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 1 0 0 0 0 0 0 0 0 0 0 0
        . 0 1 0 0 0 0 0 0 0 0 0 0
        . 0 0 1 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 1 0 0 0 0 0 0 0 0
        . 0 0 0 0 1 0 0 0 0 0 0 0
        . 0 0 0 0 0 1 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 1 0 0 0 0 0 0 0 0 0 0 0
        . 0 1 0 0 0 0 0 0 0 0 0 0
        . 0 0 1 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 1 0 0 0 0 0 0 0 0
        . 0 0 0 0 1 0 0 0 0 0 0 0
        . 0 0 0 0 0 1 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 1 0 0 0 0 0 0 0 0 0 0 0
        . 0 1 0 0 0 0 0 0 0 0 0 0
        . 0 0 1 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 1 0 0 0 0 0 0 0 0
        . 0 0 0 0 1 0 0 0 0 0 0 0
        . 0 0 0 0 0 1 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 1 0 0 0 0 0 0 0 0 0 0 0
        . 0 1 0 0 0 0 0 0 0 0 0 0
        . 0 0 1 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 1 0 0 0 0 0 0 0 0
        . 0 0 0 0 1 0 0 0 0 0 0 0
        . 0 0 0 0 0 1 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 1 0 0 0 0 0 1 0 0 0 0 0
        . 0 1 0 0 0 0 0 1 0 0 0 0
        . 0 0 1 0 0 0 0 0 1 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 1 0 0 0 0 0 1 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 0 0 0 0 0 0 0 0 0 0 0 0
        . 1 0 0 0 0 0 1 0 0 0 0 0
end

Note that there are a lot of missing values here but this is just a snapshot..

Now when I run the regressions I get the following:

. probit intreatmentgroup cleanedhigh1 cleanesempst hgsex hhtype cleanhstenr

Iteration 0:   log likelihood = -8568.7118  
Iteration 1:   log likelihood = -8553.1661  
Iteration 2:   log likelihood = -8553.1568  
Iteration 3:   log likelihood = -8553.1568  

Probit regression                               Number of obs     =     15,380
                                                LR chi2(5)        =      31.11
                                                Prob > chi2       =     0.0000
Log likelihood = -8553.1568                     Pseudo R2         =     0.0018

----------------------------------------------------------------------------------
intreatmentgroup |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
    cleanedhigh1 |   .0092955   .0041886     2.22   0.026     .0010861    .0175049
    cleanesempst |  -.0884151   .0197111    -4.49   0.000     -.127048   -.0497821
           hgsex |   .0064992   .0221838     0.29   0.770    -.0369803    .0499787
          hhtype |  -.0006135   .0013006    -0.47   0.637    -.0031627    .0019356
     cleanhstenr |   .0360111   .0186945     1.93   0.054    -.0006294    .0726515
           _cons |  -.6835616   .0551196   -12.40   0.000     -.791594   -.5755292
----------------------------------------------------------------------------------

. predict pr
(option pr assumed; Pr(intreatmentgroup))
(7945 missing values generated)

. gen ipw=1/pr*(intreatmentgroup==1) + 1/(1-pr)*(intreatmentgroup==0)
(7,945 missing values generated)

And when I then run the regressions I get again the following:

reg logcleanhwtrusi intreatmentgroup FY2007 FY2008 FY2009 FY2011 FY2012 FY2013 didFY2007 didFY2008 didFY2009 didFY2011 didFY2012 didFY2013 [pw=ipw]
(sum of wgt is   5.6528e+02)
note: FY2007 omitted because of collinearity
note: FY2008 omitted because of collinearity
note: FY2011 omitted because of collinearity
note: FY2012 omitted because of collinearity
note: didFY2007 omitted because of collinearity
note: didFY2008 omitted because of collinearity
note: didFY2011 omitted because of collinearity
note: didFY2012 omitted because of collinearity

Linear regression                               Number of obs     =        297
                                                F(5, 291)         =       5.72
                                                Prob > F          =     0.0000
                                                R-squared         =     0.1678
                                                Root MSE          =     3.4965

----------------------------------------------------------------------------------
                 |               Robust
 logcleanhwtrusi |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
intreatmentgroup |  -3.270087   1.040018    -3.14   0.002    -5.316998   -1.223176
          FY2007 |          0  (omitted)
          FY2008 |          0  (omitted)
          FY2009 |    .172953   .4500684     0.38   0.701     -.712849    1.058755
          FY2011 |          0  (omitted)
          FY2012 |          0  (omitted)
          FY2013 |   .5365671   .4414993     1.22   0.225    -.3323696    1.405504
       didFY2007 |          0  (omitted)
       didFY2008 |          0  (omitted)
       didFY2009 |    .987817   1.333974     0.74   0.460    -1.637644    3.613278
       didFY2011 |          0  (omitted)
       didFY2012 |          0  (omitted)
       didFY2013 |  -.5487583   1.747217    -0.31   0.754    -3.987542    2.890026
           _cons |   10.68515   .3564423    29.98   0.000     9.983614    11.38668
----------------------------------------------------------------------------------

and

reg logcleanhwtrusi FY2007 FY2008 FY2009 FY2011 FY2012 FY2013 didFY2007 didFY2008 didFY2009 didFY2011 didFY2012 didFY2013 [pw=ipw] if intreatmentgrou
> p ==1
(sum of wgt is   2.5707e+02)
note: FY2007 omitted because of collinearity
note: FY2008 omitted because of collinearity
note: FY2011 omitted because of collinearity
note: FY2012 omitted because of collinearity
note: didFY2007 omitted because of collinearity
note: didFY2008 omitted because of collinearity
note: didFY2009 omitted because of collinearity
note: didFY2011 omitted because of collinearity
note: didFY2012 omitted because of collinearity
note: didFY2013 omitted because of collinearity

Linear regression                               Number of obs     =         61
                                                F(2, 58)          =       0.52
                                                Prob > F          =     0.5953
                                                R-squared         =     0.0179
                                                Root MSE          =     4.4129

------------------------------------------------------------------------------
             |               Robust
logclean~usi |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      FY2007 |          0  (omitted)
      FY2008 |          0  (omitted)
      FY2009 |    1.16077   1.274749     0.91   0.366    -1.390918    3.712458
      FY2011 |          0  (omitted)
      FY2012 |          0  (omitted)
      FY2013 |  -.0121912   1.716084    -0.01   0.994    -3.447307    3.422924
   didFY2007 |          0  (omitted)
   didFY2008 |          0  (omitted)
   didFY2009 |          0  (omitted)
   didFY2011 |          0  (omitted)
   didFY2012 |          0  (omitted)
   didFY2013 |          0  (omitted)
       _cons |   7.415058   .9918062     7.48   0.000     5.429742    9.400374
------------------------------------------------------------------------------

and

reg logcleanhwtrusi FY2007 FY2008 FY2009 FY2011 FY2012 FY2013 didFY2007 didFY2008 didFY2009 didFY2011 didFY2012 didFY2013 [pw=ipw] if intreatmentgrou
> p ==0
(sum of wgt is   3.0821e+02)
note: FY2007 omitted because of collinearity
note: FY2008 omitted because of collinearity
note: FY2011 omitted because of collinearity
note: FY2012 omitted because of collinearity
note: didFY2007 omitted because of collinearity
note: didFY2008 omitted because of collinearity
note: didFY2009 omitted because of collinearity
note: didFY2011 omitted because of collinearity
note: didFY2012 omitted because of collinearity
note: didFY2013 omitted because of collinearity

Linear regression                               Number of obs     =        236
                                                F(2, 233)         =       0.88
                                                Prob > F          =     0.4180
                                                R-squared         =     0.0076
                                                Root MSE          =     2.5709

------------------------------------------------------------------------------
             |               Robust
logclean~usi |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      FY2007 |          0  (omitted)
      FY2008 |          0  (omitted)
      FY2009 |    .172953   .4483579     0.39   0.700    -.7104007    1.056307
      FY2011 |          0  (omitted)
      FY2012 |          0  (omitted)
      FY2013 |   .5365671   .4398214     1.22   0.224    -.3299679    1.403102
   didFY2007 |          0  (omitted)
   didFY2008 |          0  (omitted)
   didFY2009 |          0  (omitted)
   didFY2011 |          0  (omitted)
   didFY2012 |          0  (omitted)
   didFY2013 |          0  (omitted)
       _cons |   10.68515   .3550876    30.09   0.000     9.985552    11.38474
------------------------------------------------------------------------------

[FernandoRios]

Hi Merve,
I see the problem now. Part of this was indeed my misunderstanding. Since i wasnt looking at your data as a whole, I made some assumptions that affected my suggestions.
So first, as you can see from your mode, there are many omitted variables. I may be wrong, but i think because of this GMM is having troubles trying to fit your data.
Second. going back to your original question. Since you are trying to do a DID estimation, the interactions are already accounted for, because all explanatory variables in your model are interacted with the treatment variable. the didFY* are not needed.
My best guess is that if you drop the "didFY" variables, the model should be able to be estimated with teffects.
Perhaps this solves the problem
Fernando

[Merve Kucuk]

Hi Fernando,

Thank you very much for your recommendation. Indeed as it turns out I was not supposed to add the "did" dummies there. Furthermore, after removing them I tried to run the code on other variables of interest and it ran perfectly smooth. So I guess it must have been related to the missing values. Thanks a lot for all your help..

Kind regards
Merve

[Matilde Cappelletti]

That was unexpected. Ok, so my next suggestion is trying to replicate what teffect ipwra does by hand, and see in which step it gives you a problem.
for this:
1. Run probit (done)
2. Obtain the predicted probabilities (predict pr)
3. Define inverse probability weights for,,,say,,, ATE (gen ipw=(intreatment==1)/pr+(intreatment==0)/(1-pr))
4. Run the following regressions:

1. this is kind of a 1 step IPWRA. just to check if everything works
reg logcleanhwtrusi intreatmentgroup FY2007 FY2008 FY2009 FY2011 FY2012 FY2013 /// didFY2007 didFY2008 didFY2009 didFY2011 didFY2012 didFY2013 [pw=ipw] 2. Run separate regressions:
reg logcleanhwtrusi FY2007 FY2008 FY2009 FY2011 FY2012 FY2013 /// didFY2007 didFY2008 didFY2009 didFY2011 didFY2012 didFY2013 [pw=ipw] if intreatmentgroup ==1
reg logcleanhwtrusi FY2007 FY2008 FY2009 FY2011 FY2012 FY2013 /// didFY2007 didFY2008 didFY2009 didFY2011 didFY2012 didFY2013 [pw=ipw] if intreatmentgroup ==0

The problem may be coming from this step. Perhaps some coefficients are dropped, or have very large standard errors.

HTH
Fernando
PS. you are doing nothing wrong. what happens is that, i believe, teffects estimates the three equation model using GMM, and at some steps of the maximization, is simply having problems finding the solution.

Hi Fernando,
Sorry I'm jumping into the conversation. I am trying to estimate IPWRA by hand and your code was helpful to check was I was doing. thank you so much!

In the code above, however, I am unsure we are estimating also the first regression and not only the last two. In order to calculate ATE or ATET, isn't it enough to run the two separate regressions? Or what am I missing there?
Thanks a lot,
Matilde

[FernandoRios]

Hi Matilde,
I suggested that first regression just to see if the treatment variable was correlated with other variables Merve had on her specification.
To replicate what teffects ipwra does, you just need to later two regressions. The first one was as a verification.
HTH
Fernando

[Matilde Cappelletti]

Ok thank you for your reply!

Best,
Matilde

[Xinyi Yang]

Hi Fernando,
I am sorry to post another reply here. I have tried to use the IPWRA in STATA but failed to run it smoothly. The iteration seems endless. So I searched on Google and found this page posted by Merve. And I found your reply really helpful.
As you suggested, I have tried the first stage (probit) and it converges in 4 iterations. After that, I tried to replicate what teffect ipwra does from step 1 to step 4 with your code, and it also runs smoothly. But the single ipwra command just doesn't work. Is there any chance that you could help me with this? I would appreciate that so much.