Announcement

Collapse
No announcement yet.
X
Collapse
  • Page of 1
  • Filter
  • Time
  • Show
Clear All
new posts
  • #1

    Differences in Average Marginal Effects calculated in individual equations and an aggregate equation

    Hi, I'm estimating a DID model using count data. In my regressions I use a poisson model with weights to take into account an attrition problem in the non-response to the experiment.

    When I estimate the equation for each group of individuals, C = 1, ... 4 separately I obtain different coefficients in the average marginal effects, in comparison to when a single equation is estimated considering different terms of interaction.

    I do not understand what the difference may be due to.

    Here each of the equations individually and their AME


    Code:
    . First equation

. poisson ntbim  i.time i.treated i.did  [pw= weights ] if c==1, vce(cluster id)

Iteration 0:   log pseudolikelihood = -27353.006  
Iteration 1:   log pseudolikelihood = -27353.006  

Poisson regression                                Number of obs   =      24618
                                                  Wald chi2(3)    =     361.64
Log pseudolikelihood = -27353.006                 Prob > chi2     =     0.0000

                                  (Std. Err. adjusted for 4103 clusters in id)
------------------------------------------------------------------------------
             |               Robust
       ntbim |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      1.time |   .2601418   .0141357    18.40   0.000     .2324363    .2878474
   1.treated |   .0152871   .0078754     1.94   0.052    -.0001484    .0307226
       1.did |  -.3630388    .026564   -13.67   0.000    -.4151033   -.3109742
       _cons |   1.127303   .0060501   186.33   0.000     1.115445    1.139161
------------------------------------------------------------------------------

. margins, dydx(*)

Average marginal effects                          Number of obs   =      24618
Model VCE    : Robust

Expression   : Predicted number of events, predict()
dy/dx w.r.t. : 1.time 1.treated 1.did

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      1.time |   .8032912   .0455933    17.62   0.000     .7139299    .8926525
   1.treated |    .051304   .0264289     1.94   0.052    -.0004957    .1031036
       1.did |  -1.181889    .080278   -14.72   0.000    -1.339231   -1.024547
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. Second equation

. poisson ntbim  i.time i.treated i.did  [pw= weights ] if c==2, vce(cluster id)

Iteration 0:   log pseudolikelihood = -6626.5352  
Iteration 1:   log pseudolikelihood =  -6626.535  

Poisson regression                                Number of obs   =       3978
                                                  Wald chi2(3)    =     380.44
Log pseudolikelihood =  -6626.535                 Prob > chi2     =     0.0000

                                   (Std. Err. adjusted for 663 clusters in id)
------------------------------------------------------------------------------
             |               Robust
       ntbim |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      1.time |   .4055877   .0272316    14.89   0.000     .3522147    .4589607
   1.treated |  -.0304311   .0385052    -0.79   0.429    -.1058998    .0450377
       1.did |  -.0767001   .0384431    -2.00   0.046    -.1520471    -.001353
       _cons |   .8639051   .0294198    29.36   0.000     .8062434    .9215668
------------------------------------------------------------------------------

. margins, dydx(*)

Average marginal effects                          Number of obs   =       3978
Model VCE    : Robust

Expression   : Predicted number of events, predict()
dy/dx w.r.t. : 1.time 1.treated 1.did

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      1.time |   1.126398   .0747509    15.07   0.000     .9798891    1.272907
   1.treated |    -.09703   .1230382    -0.79   0.430    -.3381804    .1441204
       1.did |   -.243959   .1217255    -2.00   0.045    -.4825366   -.0053814
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. Third equation

. poisson ntbim  i.time i.treated i.did  [pw= weights ] if c==3, vce(cluster id)

Iteration 0:   log pseudolikelihood = -15220.496  
Iteration 1:   log pseudolikelihood = -15220.496  

Poisson regression                                Number of obs   =      10428
                                                  Wald chi2(3)    =     118.69
Log pseudolikelihood = -15220.496                 Prob > chi2     =     0.0000

                                  (Std. Err. adjusted for 1738 clusters in id)
------------------------------------------------------------------------------
             |               Robust
       ntbim |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      1.time |   .1472828    .014761     9.98   0.000     .1183519    .1762138
   1.treated |   .0084261   .0169112     0.50   0.618    -.0247193    .0415716
       1.did |  -.2329616   .0296375    -7.86   0.000      -.29105   -.1748731
       _cons |   1.930198   .0115102   167.69   0.000     1.907638    1.952758
------------------------------------------------------------------------------

. margins, dydx(*)

Average marginal effects                          Number of obs   =      10428
Model VCE    : Robust

Expression   : Predicted number of events, predict()
dy/dx w.r.t. : 1.time 1.treated 1.did

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      1.time |   1.004082   .1011709     9.92   0.000     .8057905    1.202373
   1.treated |   .0602647   .1209876     0.50   0.618    -.1768666    .2973959
       1.did |  -1.628146   .1956028    -8.32   0.000    -2.011521   -1.244772
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. Fourth equation

. poisson ntbim  i.time i.treated i.did  [pw= weights ] if c==4, vce(cluster id)

Iteration 0:   log pseudolikelihood = -36922.544  
Iteration 1:   log pseudolikelihood = -36922.544  

Poisson regression                                Number of obs   =      29148
                                                  Wald chi2(3)    =     136.72
Log pseudolikelihood = -36922.544                 Prob > chi2     =     0.0000

                                  (Std. Err. adjusted for 4858 clusters in id)
------------------------------------------------------------------------------
             |               Robust
       ntbim |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      1.time |   .0067251   .0126926     0.53   0.596    -.0181519    .0316021
   1.treated |   .0553515   .0273121     2.03   0.043     .0018207    .1088822
       1.did |  -.2756361   .0263997   -10.44   0.000    -.3273786   -.2238937
       _cons |   2.741108   .0209913   130.58   0.000     2.699966     2.78225
------------------------------------------------------------------------------

. margins, dydx(*)

Average marginal effects                          Number of obs   =      29148
Model VCE    : Robust

Expression   : Predicted number of events, predict()
dy/dx w.r.t. : 1.time 1.treated 1.did

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      1.time |   .0954102   .1799996     0.53   0.596    -.2573825     .448203
   1.treated |   .7862371   .3864334     2.03   0.042     .0288415    1.543633
       1.did |  -3.877305   .3476167   -11.15   0.000    -4.558622   -3.195989
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.
    Now, when estimating the four equations in one, I get the same coefficients but different AME's

    Code:
    . poisson ntbim  i.timeXc1 i.timeXc2 i.timeXc3 i.timeXc4 i.treatedXc1 i.treatedXc2 i.treatedXc3 i.treatedXc4 i.didXc1 i.
> didXc2 i.didXc3 i.didXc4 i.c   [pw= weights ], vce(cluster id)

Iteration 0:   log pseudolikelihood = -86164.645  
Iteration 1:   log pseudolikelihood = -86122.592  
Iteration 2:   log pseudolikelihood = -86122.581  
Iteration 3:   log pseudolikelihood = -86122.581  

Poisson regression                                Number of obs   =      68172
                                                  Wald chi2(15)   =   21288.44
Log pseudolikelihood = -86122.581                 Prob > chi2     =     0.0000

                                 (Std. Err. adjusted for 11362 clusters in id)
------------------------------------------------------------------------------
             |               Robust
       ntbim |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   1.timeXc1 |   .2601418   .0141346    18.40   0.000     .2324384    .2878452
   1.timeXc2 |   .4055878   .0272123    14.90   0.000     .3522527    .4589229
   1.timeXc3 |   .1472828   .0147574     9.98   0.000     .1183589    .1762068
   1.timeXc4 |   .0067251   .0126918     0.53   0.596    -.0181504    .0316007
1.treatedXc1 |   .0152871   .0078748     1.94   0.052    -.0001472    .0307214
1.treatedXc2 |  -.0304312   .0384778    -0.79   0.429    -.1058464    .0449839
1.treatedXc3 |   .0084261   .0169071     0.50   0.618    -.0247112    .0415635
1.treatedXc4 |   .0553515   .0273105     2.03   0.043     .0018239     .108879
    1.didXc1 |  -.3630388    .026562   -13.67   0.000    -.4150992   -.3109783
    1.didXc2 |  -.0766999   .0384158    -2.00   0.046    -.1519934   -.0014064
    1.didXc3 |  -.2329616   .0296303    -7.86   0.000    -.2910358   -.1748873
    1.didXc4 |  -.2756361   .0263982   -10.44   0.000    -.3273756   -.2238967
             |
           c |
          2  |  -.2633981   .0300149    -8.78   0.000    -.3222262     -.20457
          3  |    .802895   .0130007    61.76   0.000      .777414     .828376
          4  |   1.613805   .0218444    73.88   0.000     1.570991    1.656619
             |
       _cons |   1.127303   .0060497   186.34   0.000     1.115446     1.13916
------------------------------------------------------------------------------

. margins, dydx(*)

Average marginal effects                          Number of obs   =      68172
Model VCE    : Robust

Expression   : Predicted number of events, predict()
dy/dx w.r.t. : 1.timeXc1 1.timeXc2 1.timeXc3 1.timeXc4 1.treatedXc1 1.treatedXc2 1.treatedXc3 1.treatedXc4 1.didXc1
               1.didXc2 1.didXc3 1.didXc4 2.c 3.c 4.c

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   1.timeXc1 |   1.936844   .1189861    16.28   0.000     1.703636    2.170053
   1.timeXc2 |   3.346499    .272525    12.28   0.000      2.81236    3.880638
   1.timeXc3 |   1.053454   .1124373     9.37   0.000     .8330814    1.273827
   1.timeXc4 |   .0457593   .0864786     0.53   0.597    -.1237356    .2152543
1.treatedXc1 |   .1046128   .0542193     1.93   0.054    -.0016552    .2108807
1.treatedXc2 |  -.2040269   .2543261    -0.80   0.422     -.702497    .2944432
1.treatedXc3 |   .0575016   .1157715     0.50   0.619    -.1694064    .2844096
1.treatedXc4 |   .3813192   .1902857     2.00   0.045     .0083661    .7542722
    1.didXc1 |  -2.140394   .1345794   -15.90   0.000    -2.404165   -1.876623
    1.didXc2 |  -.5031645   .2429878    -2.07   0.038    -.9794119   -.0269171
    1.didXc3 |  -1.441417   .1654486    -8.71   0.000     -1.76569   -1.117143
    1.didXc4 |  -1.751581    .151549   -11.56   0.000    -2.048612   -1.454551
             |
           c |
          2  |  -.7609504    .078985    -9.63   0.000    -.9157582   -.6061426
          3  |   4.048494   .0886253    45.68   0.000     3.874791    4.222196
          4  |   13.21644   .3300686    40.04   0.000     12.56952    13.86337
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.
    I appreciate the suggestions you can provide. Rene
    Tags: None
  • #2
    All of this is wrong.

    First, you cannot use the -margins- command after running a model where you have hand calculated your own interaction variables. -margins- will not recognize the interaction term and will calculate something wrong and meaningless from it. So you must use factor variable notation here.

    Next, there is no such thing as the marginal effect of an interaction term. You have duped Stata into computing one for your did variable, which is because, having not used factor-variable notation, you deceived Stata about what did is. If Stata had known, it would have given you the appropriate error message. Also the statistics that -margins- calculated for the marginal effects of time and treated, while not altogether meaningless, are generally not useful and I am willing to bet are not what you think they are.

    Your combined model is mis-specified because, while you have included full interactions and the "main" effects of c, you have omitted the "main" effects of time and treated themselves.

    Here is how I would do this:

    Code:
    // FOUR SEPARATE MODELS
forvalues c = 1/4 {
    display "Model Results for c = `c'"
    poisson   ntbim i.time##i.treated if c == `c', vce(cluster id)
    margins time, dydx(treated)
    margins treated, dydx(time)
    margins time#treated
}

// COMBINED MODEL
poisson ntbim i.time##i.treated##i.c, vce(cluster id)
margins time#c, dydx(treated)
margins treated#c, dydx(time)
margins time#treated#c
    Note: Not tested, beware of typos.

    If you try these, all of these results will be meaningful and useful, and they will be consistent between the separate models and the combined model, with perhaps some slight differences due to numerical issues.

    As a side issue, if this is repeated observations on the same id's (which I'm guessing because you specified -vce(cluster id)-), shouldn't you by using -xtpoisson- instead of -poisson-?

    Comment

    • #3

      Thanks so much Clyde. I have followed your advice.

      In the case of the aggregate estimate estimated with the "poisson" command, I get the same coefficients as using "xtpoisson, fe". However, the marginal effects are not the same


      Code:
      .Poisson pooled regression

. poisson ntbim i.time##i.treated##i.c [iw=weights], vce(robust)

Iteration 0:   log pseudolikelihood = -86164.645  
Iteration 1:   log pseudolikelihood = -86122.592  
Iteration 2:   log pseudolikelihood = -86122.581  
Iteration 3:   log pseudolikelihood = -86122.581  

Poisson regression                                Number of obs   =      68172
                                                  Wald chi2(15)   =   49490.86
Log pseudolikelihood = -86122.581                 Prob > chi2     =     0.0000

--------------------------------------------------------------------------------
               |               Robust
         ntbim |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
---------------+----------------------------------------------------------------
        1.time |   .2601418   .0098192    26.49   0.000     .2408965    .2793872
     1.treated |   .0152871   .0078745     1.94   0.052    -.0001466    .0307209
               |
  time#treated |
          1 1  |  -.3630388   .0161509   -22.48   0.000    -.3946939   -.3313836
               |
             c |
            2  |  -.2633981   .0300138    -8.78   0.000     -.322224   -.2045722
            3  |    .802895   .0130003    61.76   0.000     .7774149     .828375
            4  |   1.613805   .0218436    73.88   0.000     1.570992    1.656618
               |
        time#c |
          1 2  |    .145446   .0345714     4.21   0.000     .0776872    .2132047
          1 3  |   -.112859   .0170121    -6.63   0.000     -.146202   -.0795159
          1 4  |  -.2534167   .0255607    -9.91   0.000    -.3035147   -.2033186
               |
     treated#c |
          1 2  |  -.0457184   .0392739    -1.16   0.244    -.1226939    .0312571
          1 3  |   -.006861   .0186504    -0.37   0.713    -.0434151    .0296931
          1 4  |   .0400643   .0284221     1.41   0.159     -.015642    .0957706
               |
time#treated#c |
        1 1 2  |   .2863389   .0470015     6.09   0.000     .1942177    .3784601
        1 1 3  |   .1300772   .0276726     4.70   0.000       .07584    .1843145
        1 1 4  |   .0874026   .0360028     2.43   0.015     .0168384    .1579669
               |
         _cons |   1.127303   .0060494   186.35   0.000     1.115446     1.13916
--------------------------------------------------------------------------------

. 
. margins time#c, dydx(treated)

Conditional marginal effects                      Number of obs   =      68172
Model VCE    : Robust

Expression   : Predicted number of events, predict()
dy/dx w.r.t. : 1.treated

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.treated    |
      time#c |
        0 1  |   .0475589   .0244653     1.94   0.052    -.0003922    .0955099
        0 2  |  -.0711078    .090153    -0.79   0.430    -.2478045    .1055888
        0 3  |   .0583088   .1170308     0.50   0.618    -.1710673    .2876849
        0 4  |   .8823729   .4334355     2.04   0.042      .032855    1.731891
        1 1  |  -1.176256   .0455133   -25.84   0.000     -1.26546   -1.087051
        1 2  |  -.3615712   .0731861    -4.94   0.000    -.5050133   -.2181291
        1 3  |   -1.60575   .1015442   -15.81   0.000    -1.804773   -1.406727
        1 4  |  -3.086005   .2356057   -13.10   0.000    -3.547784   -2.624227
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. margins treated#c, dydx(time)

Conditional marginal effects                      Number of obs   =      68172
Model VCE    : Robust

Expression   : Predicted number of events, predict()
dy/dx w.r.t. : 1.time

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.time       |
   treated#c |
        0 1  |   .9172858   .0361685    25.36   0.000     .8463968    .9881748
        0 2  |    1.18664   .0885154    13.41   0.000     1.013153    1.360127
        0 3  |   1.093455    .100745    10.85   0.000     .8959989    1.290912
        0 4  |   .1046189   .3664064     0.29   0.775    -.6135243    .8227622
        1 1  |  -.3065286   .0369032    -8.31   0.000    -.3788576   -.2341996
        1 2  |   .8961768   .0751584    11.92   0.000      .748869    1.043485
        1 3  |  -.5706038   .1177195    -4.85   0.000    -.8013298   -.3398779
        1 4  |  -3.863759   .3303373   -11.70   0.000    -4.511208    -3.21631
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.
      Code:
      .  xtpoisson, fe

. xtpoisson ntbim i.time##i.treated##i.c [iw=weights], fe vce(robust)

Iteration 0:   log pseudolikelihood =  -48827.09  
Iteration 1:   log pseudolikelihood = -48305.806  
Iteration 2:   log pseudolikelihood = -48301.389  
Iteration 3:   log pseudolikelihood = -48301.388  

Conditional fixed-effects Poisson regression    Number of obs      =     68172
Group variable: id                              Number of groups   =     11362

                                                Obs per group: min =         6
                                                               avg =       6.0
                                                               max =         6

                                                Wald chi2(8)       =    704.97
Log pseudolikelihood  = -48301.388              Prob > chi2        =    0.0000

                                       (Std. Err. adjusted for clustering on id)
--------------------------------------------------------------------------------
               |               Robust
         ntbim |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
---------------+----------------------------------------------------------------
        1.time |   .2601418   .0176427    14.75   0.000     .2255628    .2947208
     1.treated |          0  (omitted)
               |
  time#treated |
          1 1  |  -.3630388   .0391733    -9.27   0.000     -.439817   -.2862605
               |
             c |
            2  |          0  (omitted)
            3  |          0  (omitted)
            4  |          0  (omitted)
               |
        time#c |
          1 2  |   .1454456   .0329574     4.41   0.000     .0808504    .2100408
          1 3  |   -.112859   .0236918    -4.76   0.000     -.159294    -.066424
          1 4  |  -.2534167    .024458   -10.36   0.000    -.3013535   -.2054798
               |
     treated#c |
          1 2  |          0  (omitted)
          1 3  |          0  (omitted)
          1 4  |          0  (omitted)
               |
time#treated#c |
        1 1 2  |   .2863392   .0555823     5.15   0.000     .1773999    .3952785
        1 1 3  |   .1300772   .0529289     2.46   0.014     .0263386    .2338158
        1 1 4  |   .0874026   .0629362     1.39   0.165      -.03595    .2107553
--------------------------------------------------------------------------------

. margins time#c, dydx(treated)  predict(nu0)

Conditional marginal effects                      Number of obs   =      68172
Model VCE    : Robust

Expression   : Predicted number of events (assuming u_i=0), predict(nu0)
dy/dx w.r.t. : 1.treated

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.treated    |
      time#c |
        0 1  |          0  (omitted)
        0 2  |          0  (omitted)
        0 3  |          0  (omitted)
        0 4  |          0  (omitted)
        1 1  |  -.3948941   .0389802   -10.13   0.000    -.4712939   -.3184943
        1 2  |  -.1107614   .0570056    -1.94   0.052    -.2224903    .0009674
        1 3  |  -.2407926   .0345315    -6.97   0.000    -.3084731    -.173112
        1 4  |  -.2425365   .0392469    -6.18   0.000     -.319459   -.1656141
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. margins treated#c, dydx(time)  predict(nu0)

Conditional marginal effects                      Number of obs   =      68172
Model VCE    : Robust

Expression   : Predicted number of events (assuming u_i=0), predict(nu0)
dy/dx w.r.t. : 1.time

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.time       |
   treated#c |
        0 1  |    .297114   .0228846    12.98   0.000     .2522611    .3419669
        0 2  |   .5001835   .0417613    11.98   0.000     .4183329    .5820341
        0 3  |   .1586816   .0183217     8.66   0.000     .1227719    .1945914
        0 4  |   .0067478   .0170533     0.40   0.692    -.0266761    .0401717
        1 1  |  -.0977801   .0315556    -3.10   0.002    -.1596279   -.0359323
        1 2  |    .389422   .0388024    10.04   0.000     .3133707    .4654734
        1 3  |  -.0821109   .0292702    -2.81   0.005    -.1394794   -.0247425
        1 4  |  -.2357887   .0353483    -6.67   0.000    -.3050701   -.1665074
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

      Comment

      • #4
        So, you get the same results from the regression output of -poisson- and -xtpoisson, fe- because the interaction terms you are dealing with are all within-id effects, and -xtpoisson- is a within-id effect estimator.

        The reason that the -margins- outputs from the same commands give different results after the two regressions is that the default statistic estimated by -margins- differs. After -poisson-, -margins- estimates predictive margins and marginal effects on the actual outcome count itself (unless you specify something else). But after -xtpoisson-, the actual outcome count is not estimable, so -margins- provides predicted margins and marginal effects on xb, the linear predictor. So these are apples and oranges, and cannot be directly compared.

        Comment

        • #5

          Cyde, thank you very much again.

          Comment

          Previous Next
          Working...
          X