Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Getting Average Treatment Effects in levels after Poisson Two Way Fixed Effects regression

    Hi all,
    I am running a Poisson Two Way Fixed Effects model to estimate the effect of a policy/treatment.

    Poisson regression would produce the proportional change with respect to a counterfactual scenario without treatment, i.e. \[ (e^{\beta}-1) = \frac{\mathbb{E}[Y_{it}(1)\mid \text{Treat,Post}]-\mathbb{E}[Y_{it}(0)\mid \text{Treat,Post}]}{\mathbb{E}[Y_{it}(0)\mid \text{Treat,Post}]} \]
    where Y(1) and Y(0) are the potential outcomes with and without treatment.

    I am interested in the effect in levels, that is \[ \mathbb{E}[Y_{it}(1)\mid \text{Treat,Post}]-\mathbb{E}[Y_{it}(0)\mid \text{Treat,Post}] \]
    which theoretically it's just a matter of dividing (exp(beta)-1) by the predicted counterfactual \[ \mathbb{E}[Y_{it}(0)\mid \text{Treat,Post}] \]

    I was planning to use the margins command to do so. However, I have two questions:
    1. Is it doable/correct (as far as I understand the main issue is that the fixed effects might be imprecisely estimated)
    2.Is the right way of doing it to use the poisson regression command and not the xtpoisson? The margin command would give different estimates of the effect in levels after poisson vs xtpoisson. (because xtpoisson gets rid of the FE)

    Example code: I use hospdd, which is a Stata dataset utilised in the didregress command. I collapse the data to end up with a hospital by month panel, and finally I use the floor() function to get the outcome variable satis to be an integer. procedure is the binary treatment variables which is 1 for a selected group of hospitals and months. I am using Stata 18.
    Code:
    webuse hospdd, clear
    collapse (mean) satis procedure, by(hospital month)
    
    ge satis_int = floor(satis)
    tab satis_int
    
    collect create ex1, replace
    
    collect _r_b _r_se, tag(model[(1)]): qui poisson satis_int procedure i.hospital i.month
    
    xtset hospital month
    collect _r_b _r_se, tag(model[(2)]): qui xtpoisson satis_int procedure i.month, fe i(hospital)
    
    collect layout (colname[procedure]) (model#result[_r_b _r_se])
    The estimated coefficients of these two regression models are identical as expected

    Code:
    Collection: ex1
          Rows: colname[procedure]
       Columns: model#result[_r_b _r_se]
       Table 1: 1 x 4
    
    ----------------------------------------------------------------
                     |         (1)        (1)         (2)        (2)
                     | Coefficient Std. error Coefficient Std. error
    -----------------+----------------------------------------------
    (mean) procedure |    .2641435   .1298382    .2641435   .1298382
    ----------------------------------------------------------------
    However, the margin command would give different estimates

    Code:
    qui poisson satis_int procedure i.hospital i.month
    
    . margins, at((asobs) _all) at(procedure==0)subpop(if procedure==1) pwcompare
    
    Pairwise comparisons of predictive margins               Number of obs   = 322
    Model VCE: OIM                                           Subpop. no. obs =  72
    
    Expression: Predicted number of events, predict()
    1._at: (asobserved)    
    2._at: procedure = 0
    
    --------------------------------------------------------------
                 |            Delta-method         Unadjusted
                 |   Contrast   std. err.     [95% conf. interval]
    -------------+------------------------------------------------
             _at |
         2 vs 1  |  -.8995296   .4139294     -1.710816    -.088243
    --------------------------------------------------------------
    
    .
    . qui xtpoisson satis_int procedure i.month, fe i(hospital)
    
    . margins, at((asobs) _all) at(procedure==0)subpop(if procedure==1) pwcompare
    
    Pairwise comparisons of predictive margins               Number of obs   = 322
    Model VCE: OIM                                           Subpop. no. obs =  72
    
    Expression: Linear prediction, predict()
    1._at: (asobserved)    
    2._at: procedure = 0
    
    --------------------------------------------------------------
                 |            Delta-method         Unadjusted
                 |   Contrast   std. err.     [95% conf. interval]
    -------------+------------------------------------------------
             _at |
         2 vs 1  |  -.2641435   .1298382     -.5186218   -.0096652
    --------------------------------------------------------------
    In particular, margins after xtpoisson simply returns the estimated coefficients, which is mentioned in the help. I know similar questions have been posted here https://www.statalist.org/forums/for...fter-xtpoisson and @JoaoSantosSilva has commented on it in the past, but wanted to double check my reasoning is correct.
    Cheers,
    Mario
    Last edited by mariofiorini; 27 Nov 2023, 22:34.

  • #2
    Dear mariofiorini,

    In short, do not use margins after xtpoisson. If your model is reasonably small (not many fixed effects) and you have no convergence problems, you should be able to estimate the model with poisson and use margins after that. Otherwise, you can estimate the model with ppmlhdfe and compute what you want based on the results (in any case, I would compute the results "manually" just to check that margins is doing what it is intended to do). Keep in mind that, whatever the method you use, the effects you estimate will depend on the estimated FEs and can be noisy, unless each category has a large enough sample.

    Best wishes,

    Joao

    Comment


    • #3
      Dear Joao Santos Silva thanks for the reply. To summarise if anyone else is interested, poisson, xtpoisson and ppmlhdfe will give you the same estimates of the proportional effects (although I did not manage to get the same S.E. - vce(cluster hospital) returned an error message in xtpoisson). However, the three methods will give different results when using margins. From Joao's previous post I understood that xtpoisson would do that and it is to be avoided - xtpoisson will simply return the poisson coefficient. But ppmlhdfe also gave a different result (linking @Sergio Correia). Computing the ATT manually matched the result from using margins after poisson.

      COMPARING POISSON ESTIMATES:

      Code:
      cls 
      
      webuse hospdd, clear
      collapse (mean) satis procedure, by(hospital month)
      
      ge satis_int = floor(satis)
      tab satis_int
      
      collect create ex1, replace
      
      collect, tag(model[(poisson)]): quietly:  poisson satis_int procedure i.hospital i.month,  vce(cluster hospital) 
      
      xtset hospital month
      collect, tag(model[(xtpoisson)]): quietly: xtpoisson satis_int procedure i.month, fe i(hospital) robust
      
      collect, tag(model[(ppmlhdfe)]): quietly: ppmlhdfe satis_int procedure, absorb(month hospital) vce(cluster hospital) 
      
      collect layout (result[_r_b _r_se]) (model#colname[procedure]) 
      
      Collection: ex1
            Rows: result[_r_b _r_se]
         Columns: model#colname[procedure]
         Table 1: 2 x 3
      
      ----------------------------------------------------------------
                  |        (poisson)      (xtpoisson)       (ppmlhdfe)
                  | (mean) procedure (mean) procedure (mean) procedure
      ------------+---------------------------------------------------
      Coefficient |         .2641435         .2641435         .2641435
      Std. error  |         .0273645         .0270654         .0273645
      ----------------------------------------------------------------
      COMPARING MARGINS ESTIMATES:
      Code:
      qui xtpoisson satis_int procedure i.month, fe i(hospital)
      margins, at((asobs) _all) at(procedure==0) subpop(if procedure==1) pwcompare
      
      Pairwise comparisons of predictive margins               Number of obs   = 322
      Model VCE: OIM                                           Subpop. no. obs =  72
      
      Expression: Linear prediction, predict()
      1._at: (asobserved)    
      2._at: procedure = 0
      
      --------------------------------------------------------------
                   |            Delta-method         Unadjusted
                   |   Contrast   std. err.     [95% conf. interval]
      -------------+------------------------------------------------
               _at |
           2 vs 1  |  -.2641435   .1298382     -.5186218   -.0096652
      --------------------------------------------------------------
      
      
      qui ppmlhdfe satis_int procedure, absorb(month hospital) d
      margins, at((asobs) _all) at(procedure==0) subpop(if procedure==1) pwcompare
      
      Pairwise comparisons of predictive margins               Number of obs   = 322
      Model VCE: Robust                                        Subpop. no. obs =  72
      
      Expression: Predicted mean of satis_int, predict()
      1._at: (asobserved)    
      2._at: procedure = 0
      
      --------------------------------------------------------------
                   |            Delta-method         Unadjusted
                   |   Contrast   std. err.     [95% conf. interval]
      -------------+------------------------------------------------
               _at |
           2 vs 1  |  -1.198227   .0638356     -1.323343   -1.073112
      --------------------------------------------------------------
      
      
      qui poisson satis_int procedure i.hospital i.month
      margins, at((asobs) _all) at(procedure==0) subpop(if procedure==1) pwcompare
      
      Pairwise comparisons of predictive margins               Number of obs   = 322
      Model VCE: OIM                                           Subpop. no. obs =  72
      
      Expression: Predicted number of events, predict()
      1._at: (asobserved)    
      2._at: procedure = 0
      
      --------------------------------------------------------------
                   |            Delta-method         Unadjusted
                   |   Contrast   std. err.     [95% conf. interval]
      -------------+------------------------------------------------
               _at |
           2 vs 1  |  -.8995296   .4139294     -1.710816    -.088243
      --------------------------------------------------------------
      COMPUTING THE ATT MANUALLY:

      Code:
      qui poisson satis_int procedure i.hospital i.month
      cap drop procedureN
      gen procedureN = procedure
      replace procedure = 0 // so predict will get the counterfactual Y(0)
      cap drop y0
      predict y0 
      
      su y0 if procedureN == 1
      local Ey0 = r(mean) // mean of the counterfactual Y(0)
      su satis_int if procedureN == 1
      local Ey1 = r(mean) // mean of Y(1)
      
      di (`Ey1'-`Ey0')
      .89952957
      Note that if I had used ppmlhdfe instead of poisson when computing manually I would still get the same result. So, as far as I can tell, it is the combination of ppmlhdfe+margins that is not working well.

      Comment


      • #4
        Dear mariofiorini,

        Thanks for this. I am not surprised that ppmlhdfe does not work well with margins; I do not think it was designed to be used in that way. So, if you have a lot of FEs, my suggestion would be to estimate by ppmlhdfe, and compute the effects by hand.

        Best wishes,

        Joao

        Comment


        • #5
          Actually
          i was just working on adapting ppmlhdfe for jwdid.
          the latest version of ppmlhdfe does work with margins if you also include the option ā€œdā€

          Comment


          • #6
            FernandoRios I have updated ppmlhdfe but still did not work in the example above. I do include the option `d'.

            Comment

            Working...
            X