Need Help with Differences-in-Differences model.

[Issy Ojalvo]

Hello all,

I've been trying to preform this statistical analysis for a while, but its been tough as I lack the proper stats and stata background so any help would be appreciated. Essentially my project looks at the different variances of a apology healthcare law in order to see which one has been most effective at lowering payment costs and incidences of malpractice. My variables in general are: malpayment2 (the payment ammount accounted for with inflation), before(0= the 1-3 years before treatment, 1 = 3-5 years after treatment), before2(for the states that don't have the law I used the 0 = average 1-3 years before and 1 = the average of 3-5 years after, this is a control), apology(1=state has this healthcare law, 0= state doesn't have this healthcare law), and finally I have three other variants of this law: genexp(1=state has specific type of apology law, 0= has apology law but not this specific one, .= state has no apology law), and same idea for protfault, and disclos variables.

Now if you're still able to follow along, I think a differences in differences model would be most effective at seeing which one of these variances of the law (apology) has been most effective at lowering malpractice costs (malpayment2) and the observations. I downloaded diff with "ssc install diff" and have been messing around with it but I feel like it is still not giving me the results I want. Should I maybe use a DDD model so I could compare the control group to the treatment group to the specific variances perhaps? Any help would be appreciated.

[Clyde Schechter]

It is likely that -diff- is giving you spurious results because it sounds like your data is not properly structured for the analysis. You can't do this with separate before variables for the two groups. There has to be a single variable that distinguishes before and after the onset of treatment in both groups. This sounds like one of these variant DID designs where the time of onset of treatment in the treatment group varies. And it sounds like you have done something to identify a corresponding point in time among the controls. So it sounds like you need to put together your two before variables. For example:

gen pre_post = before if apology == 1
replace pre_post = before2 if apology == 0

Then your main DID analysis for the effect of apology will be:

some_regression_command outcome_variable i.apology##i.pre_post // AND POSSIBLY SOME COVARIATES
margins apology#pre_post
margins apology, dydx(pre_post)

Then you have the additional complication of some variations on the apology law. Again, that variable appears not be set up in a useful way In particular, missing values are only useful if your intention is to exclude those observations from analysis. So I would revise the variable genexp so that it codes 0 for no apology law, 1 for the specific type of apology law, and 2 for some other kind of apology law). The analysis is then similar:

some_regression_command outcome_variable i.genexp##i.pre_post
margins genexp#pre_post
magins genexp, dydx(pre_post)

The -diff- program is a user-written convenience command. Personally, I've never thought it to be particularly helpful: it's more effort and time to learn the different options and what they mean than to just write a simple regression command and some -margins- commands to get the results. Such things are, inevitably, in the eye of the beholder.

[Issy Ojalvo]

Thanks so much Clyde Schechter. I hope this post reaches you. I did everything you told me to and my data is finally coming along! One last thing, would you mind interpreting the data just to make sure that has some value. Some of the results were really surprising, like payments actually went up in 2/3 variances of the law. I attached a word document that contains pictures of all of it. There might have been a better way to export it but oh well.

[Clyde Schechter]

No offense, but I never download Microsoft Office documents from somebody I don't know. They can contain active malware (that you might not even be aware of if somebody has infected your computer). My livelihood depends on my computer!

If you are looking to show Stata output, just copy the output from Stata's Results window (or your log file) and paste it between code delimiters here on the Forum. It will come out nicely readable. (If you don't know about code delimiters, read FAQ #12. If you are trying to show graphs, I suggest you -graph export- them as png files and then post those.

[Issy Ojalvo]

I understand the concern, no worries. Here you go, I did it once with the whole data set, and the second one is with the major outliers removed.

. reg payment2 i.apology##i.pre_post

      Source |       SS           df       MS      Number of obs   =    62,701
-------------+----------------------------------   F(3, 62697)     =    229.23
       Model |  2.3983e+14         3  7.9945e+13   Prob > F        =    0.0000
    Residual |  2.1866e+16    62,697  3.4875e+11   R-squared       =    0.0108
-------------+----------------------------------   Adj R-squared   =    0.0108
       Total |  2.2106e+16    62,700  3.5256e+11   Root MSE        =    5.9e+05

----------------------------------------------------------------------------------
        payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
       1.apology |  -121169.8   6552.365   -18.49   0.000    -134012.4   -108327.1
      1.pre_post |  -6355.206   8667.657    -0.73   0.463    -23343.83    10633.42
                 |
apology#pre_post |
            1 1  |  -23663.37   10420.61    -2.27   0.023    -44087.79   -3238.955
                 |
           _cons |   400512.3    5415.18    73.96   0.000     389898.6    411126.1
----------------------------------------------------------------------------------

. 
. margins apology#pre_post

Adjusted predictions                            Number of obs     =     62,701
Model VCE    : OLS

Expression   : Linear prediction, predict()

----------------------------------------------------------------------------------
                 |            Delta-method
                 |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
apology#pre_post |
            0 0  |   400512.3    5415.18    73.96   0.000     389898.6    411126.1
            0 1  |   394157.1   6767.872    58.24   0.000     380892.1    407422.2
            1 0  |   279342.5   3689.081    75.72   0.000     272111.9    286573.1
            1 1  |     249324    4455.51    55.96   0.000     240591.2    258056.8
----------------------------------------------------------------------------------

. 
. margins apology, dydx(pre_post)

Conditional marginal effects                    Number of obs     =     62,701
Model VCE    : OLS

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
     apology |
          0  |  -6355.206   8667.657    -0.73   0.463    -23343.83    10633.42
          1  |  -30018.58   5784.539    -5.19   0.000    -41356.29   -18680.87
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. reg payment2 i.apology##i.pre_post, robust

Linear regression                               Number of obs     =     62,701
                                                F(3, 62697)       =     241.66
                                                Prob > F          =     0.0000
                                                R-squared         =     0.0108
                                                Root MSE          =     5.9e+05

----------------------------------------------------------------------------------
                 |               Robust
        payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
       1.apology |  -121169.8   6642.922   -18.24   0.000    -134189.9   -108149.7
      1.pre_post |  -6355.206   9558.172    -0.66   0.506    -25089.24    12378.83
                 |
apology#pre_post |
            1 1  |  -23663.37   10940.65    -2.16   0.031    -45107.07   -2219.681
                 |
           _cons |   400512.3   5336.466    75.05   0.000     390052.8    410971.8
----------------------------------------------------------------------------------

. reg payment2 i.genexp##i.pre_post

      Source |       SS           df       MS      Number of obs   =    62,701
-------------+----------------------------------   F(5, 62695)     =    181.54
       Model |  3.1548e+14         5  6.3096e+13   Prob > F        =    0.0000
    Residual |  2.1790e+16    62,695  3.4756e+11   R-squared       =    0.0143
-------------+----------------------------------   Adj R-squared   =    0.0142
       Total |  2.2106e+16    62,700  3.5256e+11   Root MSE        =    5.9e+05

---------------------------------------------------------------------------------
       payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------+----------------------------------------------------------------
         genexp |
             1  |  -146812.3   7163.364   -20.49   0.000    -160852.5   -132772.1
             2  |  -87351.56   8133.881   -10.74   0.000      -103294   -71409.14
                |
     1.pre_post |  -6279.483   8760.055    -0.72   0.473    -23449.21    10890.24
                |
genexp#pre_post |
           1 1  |  -42187.38    11417.5    -3.69   0.000     -64565.7   -19809.06
           2 1  |   1106.607   12750.78     0.09   0.931    -23884.95    26098.16
                |
          _cons |   404024.2   5479.409    73.73   0.000     393284.5    414763.8
---------------------------------------------------------------------------------

. 
. margins genexp#pre_post

Adjusted predictions                            Number of obs     =     62,701
Model VCE    : OLS

Expression   : Linear prediction, predict()

---------------------------------------------------------------------------------
                |            Delta-method
                |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------+----------------------------------------------------------------
genexp#pre_post |
           0 0  |   404024.2   5479.409    73.73   0.000     393284.5    414763.8
           0 1  |   397744.7   6834.811    58.19   0.000     384348.5      411141
           1 0  |   257211.9   4614.093    55.74   0.000     248168.2    266255.5
           1 1  |     208745   5686.025    36.71   0.000     197600.4    219889.6
           2 0  |   316672.6    6011.33    52.68   0.000     304890.4    328454.9
           2 1  |   311499.8   7050.376    44.18   0.000       297681    325318.5
---------------------------------------------------------------------------------

. 
. margins genexp, dydx(pre_post)

Conditional marginal effects                    Number of obs     =     62,701
Model VCE    : OLS

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
      genexp |
          0  |  -6279.483   8760.055    -0.72   0.473    -23449.21    10890.24
          1  |  -48466.86   7322.618    -6.62   0.000    -62819.21   -34114.52
          2  |  -5172.876   9265.198    -0.56   0.577    -23332.68    12986.93
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. reg payment2 i.protfault##i.pre_post

      Source |       SS           df       MS      Number of obs   =    62,701
-------------+----------------------------------   F(5, 62695)     =    154.17
       Model |  2.6849e+14         5  5.3698e+13   Prob > F        =    0.0000
    Residual |  2.1837e+16    62,695  3.4831e+11   R-squared       =    0.0121
-------------+----------------------------------   Adj R-squared   =    0.0121
       Total |  2.2106e+16    62,700  3.5256e+11   Root MSE        =    5.9e+05

------------------------------------------------------------------------------------
          payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------------+----------------------------------------------------------------
         protfault |
                1  |  -82009.48   11160.49    -7.35   0.000    -103884.1    -60134.9
                2  |  -127686.4   6718.661   -19.00   0.000      -140855   -114517.8
                   |
        1.pre_post |  -6355.206   8662.113    -0.73   0.463    -23332.96    10622.55
                   |
protfault#pre_post |
              1 1  |   23066.94    17501.6     1.32   0.188    -11236.23    57370.11
              2 1  |  -31804.31   10681.55    -2.98   0.003    -52740.17   -10868.45
                   |
             _cons |   400512.3   5411.717    74.01   0.000     389905.4    411119.3
------------------------------------------------------------------------------------

. margins protfault, dydx(pre_post)

Conditional marginal effects                    Number of obs     =     62,701
Model VCE    : OLS

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
   protfault |
          0  |  -6355.206   8662.113    -0.73   0.463    -23332.96    10622.55
          1  |   16711.74   15207.69     1.10   0.272    -13095.37    46518.84
          2  |  -38159.52   6250.066    -6.11   0.000    -50409.66   -25909.37
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. margins protfault#pre_post

Adjusted predictions                            Number of obs     =     62,701
Model VCE    : OLS

Expression   : Linear prediction, predict()

------------------------------------------------------------------------------------
                   |            Delta-method
                   |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------------+----------------------------------------------------------------
protfault#pre_post |
              0 0  |   400512.3   5411.717    74.01   0.000     389905.4    411119.3
              0 1  |   394157.1   6763.544    58.28   0.000     380900.6    407413.7
              1 0  |   318502.8   9760.626    32.63   0.000       299372    337633.7
              1 1  |   335214.6   11662.08    28.74   0.000     312356.9    358072.3
              2 0  |   272825.9   3981.674    68.52   0.000     265021.8      280630
              2 1  |   234666.4   4817.634    48.71   0.000     225223.8      244109
------------------------------------------------------------------------------------

. reg payment2 i.disclos##i.pre_post

      Source |       SS           df       MS      Number of obs   =    62,701
-------------+----------------------------------   F(5, 62695)     =    115.87
       Model |  2.0240e+14         5  4.0481e+13   Prob > F        =    0.0000
    Residual |  2.1903e+16    62,695  3.4936e+11   R-squared       =    0.0092
-------------+----------------------------------   Adj R-squared   =    0.0091
       Total |  2.2106e+16    62,700  3.5256e+11   Root MSE        =    5.9e+05

----------------------------------------------------------------------------------
        payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
         disclos |
              1  |  -134056.1   7255.521   -18.48   0.000    -148276.9   -119835.2
              2  |  -66130.02   7489.428    -8.83   0.000    -80809.31   -51450.73
                 |
      1.pre_post |  -33178.35   7524.587    -4.41   0.000    -47926.55   -18430.15
                 |
disclos#pre_post |
            1 1  |   5529.588   11515.06     0.48   0.631    -17039.94    28099.12
            2 1  |    25891.1   11745.84     2.20   0.028     2869.236    48912.96
                 |
           _cons |   377109.3   4722.045    79.86   0.000     367854.1    386364.5
----------------------------------------------------------------------------------

. margins disclos#pre_post

Adjusted predictions                            Number of obs     =     62,701
Model VCE    : OLS

Expression   : Linear prediction, predict()

----------------------------------------------------------------------------------
                 |            Delta-method
                 |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
disclos#pre_post |
            0 0  |   377109.3   4722.045    79.86   0.000     367854.1    386364.5
            0 1  |     343931   5858.472    58.71   0.000     332448.4    355413.6
            1 0  |   243053.3   5508.618    44.12   0.000     232256.3    253850.2
            1 1  |   215404.5   6755.165    31.89   0.000     202164.4    228644.6
            2 0  |   310979.3   5813.245    53.49   0.000     299585.3    322373.3
            2 1  |   303692.1   6895.756    44.04   0.000     290176.4    317207.7
----------------------------------------------------------------------------------

. margins disclos, dydx(pre_post)

Conditional marginal effects                    Number of obs     =     62,701
Model VCE    : OLS

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
     disclos |
          0  |  -33178.35   7524.587    -4.41   0.000    -47926.55   -18430.15
          1  |  -27648.76   8716.486    -3.17   0.002    -44733.09   -10564.43
          2  |  -7287.254   9019.162    -0.81   0.419    -24964.83    10390.32
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

[Issy Ojalvo]

Here I had the outliers removed. Which was about 14,000 out of 300,000 observations.

. reg payment2 i.apology##i.pre_post

      Source |       SS           df       MS      Number of obs   =    60,403
-------------+----------------------------------   F(3, 60399)     =    508.42
       Model |  1.1344e+14         3  3.7812e+13   Prob > F        =    0.0000
    Residual |  4.4920e+15    60,399  7.4372e+10   R-squared       =    0.0246
-------------+----------------------------------   Adj R-squared   =    0.0246
       Total |  4.6054e+15    60,402  7.6246e+10   Root MSE        =    2.7e+05

----------------------------------------------------------------------------------
        payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
       1.apology |  -92569.59   3105.952   -29.80   0.000    -98657.27   -86481.92
      1.pre_post |  -1612.934   4111.884    -0.39   0.695     -9672.24    6446.372
                 |
apology#pre_post |
            1 1  |  -3383.698   4922.679    -0.69   0.492    -13032.17    6264.769
                 |
           _cons |   304466.9   2576.999   118.15   0.000       299416    309517.9
----------------------------------------------------------------------------------

. margins apology#pre_post

Adjusted predictions                            Number of obs     =     60,403
Model VCE    : OLS

Expression   : Linear prediction, predict()

----------------------------------------------------------------------------------
                 |            Delta-method
                 |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
apology#pre_post |
            0 0  |   304466.9   2576.999   118.15   0.000       299416    309517.9
            0 1  |     302854   3204.164    94.52   0.000     296573.8    309134.2
            1 0  |   211897.3   1733.785   122.22   0.000     208499.1    215295.6
            1 1  |   206900.7    2078.26    99.55   0.000     202827.3    210974.1
----------------------------------------------------------------------------------

. margins apology, dydx(pre_post)

Conditional marginal effects                    Number of obs     =     60,403
Model VCE    : OLS

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
     apology |
          0  |  -1612.934   4111.884    -0.39   0.695     -9672.24    6446.372
          1  |  -4996.632   2706.507    -1.85   0.065    -10301.39    308.1291
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. reg payment2 i.genexp##i.pre_post

      Source |       SS           df       MS      Number of obs   =    60,403
-------------+----------------------------------   F(5, 60397)     =    409.02
       Model |  1.5084e+14         5  3.0168e+13   Prob > F        =    0.0000
    Residual |  4.4546e+15    60,397  7.3755e+10   R-squared       =    0.0328
-------------+----------------------------------   Adj R-squared   =    0.0327
       Total |  4.6054e+15    60,402  7.6246e+10   Root MSE        =    2.7e+05

---------------------------------------------------------------------------------
       payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------+----------------------------------------------------------------
         genexp |
             1  |  -115400.3    3382.07   -34.12   0.000    -122029.2   -108771.4
             2  |  -58293.85   3840.981   -15.18   0.000    -65822.19   -50765.52
                |
     1.pre_post |  -1763.444   4147.692    -0.43   0.671    -9892.934    6366.045
                |
genexp#pre_post |
           1 1  |   -6404.53   5367.983    -1.19   0.233    -16925.79    4116.734
           2 1  |  -1452.585   6005.972    -0.24   0.809    -13224.31    10319.14
                |
          _cons |   306490.7   2602.445   117.77   0.000     301389.9    311591.5
---------------------------------------------------------------------------------

. margins genexp#pre_post

Adjusted predictions                            Number of obs     =     60,403
Model VCE    : OLS

Expression   : Linear prediction, predict()

---------------------------------------------------------------------------------
                |            Delta-method
                |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------+----------------------------------------------------------------
genexp#pre_post |
           0 0  |   306490.7   2602.445   117.77   0.000     301389.9    311591.5
           0 1  |   304727.3   3229.648    94.35   0.000     298397.2    311057.4
           1 0  |   191090.4   2160.017    88.47   0.000     186856.8    195324.1
           1 1  |   182922.4   2635.568    69.41   0.000     177756.7    188088.2
           2 0  |   248196.9   2824.962    87.86   0.000     242659.9    253733.8
           2 1  |   244980.9   3299.689    74.24   0.000     238513.5    251448.3
---------------------------------------------------------------------------------

. margins genexp, dydx(pre_post)

Conditional marginal effects                    Number of obs     =     60,403
Model VCE    : OLS

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
      genexp |
          0  |  -1763.444   4147.692    -0.43   0.671    -9892.934    6366.045
          1  |  -8167.974   3407.622    -2.40   0.017    -14846.93   -1489.023
          2  |  -3216.029   4343.772    -0.74   0.459    -11729.84    5297.778
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. reg payment2 i.protfault##i.pre_post

      Source |       SS           df       MS      Number of obs   =    60,403
-------------+----------------------------------   F(5, 60397)     =    351.56
       Model |  1.3024e+14         5  2.6049e+13   Prob > F        =    0.0000
    Residual |  4.4752e+15    60,397  7.4096e+10   R-squared       =    0.0283
-------------+----------------------------------   Adj R-squared   =    0.0282
       Total |  4.6054e+15    60,402  7.6246e+10   Root MSE        =    2.7e+05

------------------------------------------------------------------------------------
          payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------------+----------------------------------------------------------------
         protfault |
                1  |  -55440.52   5273.018   -10.51   0.000    -65775.65   -45105.38
                2  |  -98681.46   3178.702   -31.04   0.000    -104911.7   -92451.19
                   |
        1.pre_post |  -1612.934   4104.252    -0.39   0.694    -9657.282    6431.414
                   |
protfault#pre_post |
              1 1  |   21832.18   8250.342     2.65   0.008     5661.488    38002.88
              2 1  |  -7697.882   5035.426    -1.53   0.126    -17567.33    2171.568
                   |
             _cons |   304466.9   2572.216   118.37   0.000     299425.4    309508.5
------------------------------------------------------------------------------------

. margins protfault#pre_post

Adjusted predictions                            Number of obs     =     60,403
Model VCE    : OLS

Expression   : Linear prediction, predict()

------------------------------------------------------------------------------------
                   |            Delta-method
                   |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------------+----------------------------------------------------------------
protfault#pre_post |
              0 0  |   304466.9   2572.216   118.37   0.000     299425.4    309508.5
              0 1  |     302854   3198.217    94.69   0.000     296585.5    309122.5
              1 0  |   249026.4   4603.088    54.10   0.000     240004.3    258048.5
              1 1  |   269245.7   5480.404    49.13   0.000     258504.1    279987.3
              2 0  |   205785.5   1867.579   110.19   0.000       202125    209445.9
              2 1  |   196474.7   2241.154    87.67   0.000       192082    200867.3
------------------------------------------------------------------------------------

. margins protfault, dydx(pre_post)

Conditional marginal effects                    Number of obs     =     60,403
Model VCE    : OLS

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
   protfault |
          0  |  -1612.934   4104.252    -0.39   0.694    -9657.282    6431.414
          1  |   20219.25   7157.042     2.83   0.005     6191.425    34247.08
          2  |  -9310.816   2917.297    -3.19   0.001    -15028.73   -3592.904
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. reg payment2 i.disclos##i.pre_post

      Source |       SS           df       MS      Number of obs   =    60,403
-------------+----------------------------------   F(5, 60397)     =    270.63
       Model |  1.0092e+14         5  2.0184e+13   Prob > F        =    0.0000
    Residual |  4.5045e+15    60,397  7.4581e+10   R-squared       =    0.0219
-------------+----------------------------------   Adj R-squared   =    0.0218
       Total |  4.6054e+15    60,402  7.6246e+10   Root MSE        =    2.7e+05

----------------------------------------------------------------------------------
        payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
         disclos |
              1  |  -102053.7   3424.749   -29.80   0.000    -108766.2   -95341.15
              2  |  -35221.66    3536.02    -9.96   0.000    -42152.27   -28291.05
                 |
      1.pre_post |  -12185.86   3556.502    -3.43   0.001    -19156.61   -5215.103
                 |
disclos#pre_post |
            1 1  |   14103.99   5407.715     2.61   0.009     3504.854    24703.13
            2 1  |   9558.209   5529.319     1.73   0.084    -1279.276    20395.69
                 |
           _cons |   282102.2   2241.506   125.85   0.000     277708.8    286495.5
----------------------------------------------------------------------------------

. margins disclos#pre_post

Adjusted predictions                            Number of obs     =     60,403
Model VCE    : OLS

Expression   : Linear prediction, predict()

----------------------------------------------------------------------------------
                 |            Delta-method
                 |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
disclos#pre_post |
            0 0  |   282102.2   2241.506   125.85   0.000     277708.8    286495.5
            0 1  |   269916.3   2761.224    97.75   0.000     264504.3    275328.3
            1 0  |   180048.5   2589.315    69.54   0.000     174973.4    185123.6
            1 1  |   181966.7   3144.857    57.86   0.000     175802.7    188130.6
            2 0  |   246880.5   2734.792    90.27   0.000     241520.3    252240.7
            2 1  |   244252.9   3231.962    75.57   0.000     237918.2    250587.5
----------------------------------------------------------------------------------

. margins disclos, dydx(pre_post)

Conditional marginal effects                    Number of obs     =     60,403
Model VCE    : OLS

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
     disclos |
          0  |  -12185.86   3556.502    -3.43   0.001    -19156.61   -5215.103
          1  |   1918.135   4073.657     0.47   0.638    -6066.245    9902.516
          2  |   -2627.65   4233.753    -0.62   0.535    -10925.82     5670.52
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

.

[Clyde Schechter]

The regressions appear to have been properly done and -margins- properly applied.

Be a little bit cautious about how you interpret these findings. For example, in

. reg payment2 i.genexp##i.pre_post

      Source |       SS           df       MS      Number of obs   =    62,701
-------------+----------------------------------   F(5, 62695)     =    181.54
       Model |  3.1548e+14         5  6.3096e+13   Prob > F        =    0.0000
    Residual |  2.1790e+16    62,695  3.4756e+11   R-squared       =    0.0143
-------------+----------------------------------   Adj R-squared   =    0.0142
       Total |  2.2106e+16    62,700  3.5256e+11   Root MSE        =    5.9e+05

---------------------------------------------------------------------------------
       payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------+----------------------------------------------------------------
         genexp |
             1  |  -146812.3   7163.364   -20.49   0.000    -160852.5   -132772.1
             2  |  -87351.56   8133.881   -10.74   0.000      -103294   -71409.14
                |
     1.pre_post |  -6279.483   8760.055    -0.72   0.473    -23449.21    10890.24
                |
genexp#pre_post |
           1 1  |  -42187.38    11417.5    -3.69   0.000     -64565.7   -19809.06
           2 1  |   1106.607   12750.78     0.09   0.931    -23884.95    26098.16
                |
          _cons |   404024.2   5479.409    73.73   0.000     393284.5    414763.8
---------------------------------------------------------------------------------

. 
. margins genexp#pre_post

Adjusted predictions                            Number of obs     =     62,701
Model VCE    : OLS

Expression   : Linear prediction, predict()

---------------------------------------------------------------------------------
                |            Delta-method
                |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------+----------------------------------------------------------------
genexp#pre_post |
           0 0  |   404024.2   5479.409    73.73   0.000     393284.5    414763.8
           0 1  |   397744.7   6834.811    58.19   0.000     384348.5      411141
           1 0  |   257211.9   4614.093    55.74   0.000     248168.2    266255.5
           1 1  |     208745   5686.025    36.71   0.000     197600.4    219889.6
           2 0  |   316672.6    6011.33    52.68   0.000     304890.4    328454.9
           2 1  |   311499.8   7050.376    44.18   0.000       297681    325318.5
---------------------------------------------------------------------------------

. 
. margins genexp, dydx(pre_post)

Conditional marginal effects                    Number of obs     =     62,701
Model VCE    : OLS

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
      genexp |
          0  |  -6279.483   8760.055    -0.72   0.473    -23449.21    10890.24
          1  |  -48466.86   7322.618    -6.62   0.000    -62819.21   -34114.52
          2  |  -5172.876   9265.198    -0.56   0.577    -23332.68    12986.93
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level

If you look at the -margins, dydx()- output, you can see that the expected value of payment2 is decreased following the policy date in all three groups: all three marginal effects of pre_post are negative. However, you can also see that the effect is less negative for genexp level 2 than for genexp level 0. This is why the interaction coefficient for 2.genexp#1.pre_post is positive. This should not be interpreted as saing that payment2 increases in condition genexp = 2. It does not, it just decreases less than it does in the genexp = 0 condition.

I will just add that removing outliers from analysis is a practice I discourage. At best it materially limits the applicability of your conclusions to entities that are not outliers on other variables. At worst, it results in a severely biased sample. When cleaning data, it makes sense to identify outliers, and it often makes sense to then check back with the original data source to make sure that the outlying values are correct. And if on checking they are found to be errors, then, of course the errors should be corrected, if possible, or removed. But there is no reason to presume that outlying values are errors. They are just as legitimate as non-lying values and should be included in analyses. It is sometimes true that outlying values exert a large degree of influence on the results of certain types of models. That is also worth knowing about--and one way of doing that is to rerun the model without them. And if that is found, it is sometimes worth either modeling the outliers separately, or creating a more flexible model that regularizes their effect or separates out their effects. But in general, results from analyses with outliers removed should not be reported in isolation or as a principal finding.

[Issy Ojalvo]

Thanks so much again Clyde, I'm going to acknowledge you in the paper I write. Since the results were similar with the outliers included, I'm just going to keep them. Final question, I also want to be able to tell if the differences in observations of lawsuits are statistically significant from before and after implementation of the laws as well. Each payment amount corresponds to a different lawsuit, so its naturally built into the system and I can see the differences in observations using other methods. But I'm not sure how to do any statistical analysis on significance Essentially, I wonder if I can do a differences in differences model but with the variable tested being the number of observations itself. Or perhaps a paired t-test or something might be better suited for that. I'm just not sure how I would set that variable up, or if I even need to set up a new variable. Thanks again.

[Clyde Schechter]

I think the use of the number of observations as an outcome is problematic. First, if the number were to actually drop to zero for some entity, then they would not appear in your data at all. So you have a biased sample in that regard. All of your results are based on analyses of lawsuits that occurred: you can't say anything about lawsuits that didn't occur on this kind of data. The other issue is that a simple count of lawsuits would be misleading because, all else being equal, you would expect more lawsuits in states/cities/counties/whatever that have larger populations. You would expect more lawsuits in hospitals that handle a larger volume of admissions. Etc. In other words, you would need not a count of incident lawsuits but a rate of lawsuits (i.e. the number of lawsuits divided by a denominator that reflects population or number of "opportunities" for a lawsuit to arise) in order to draw meaningful conclusions.

[Issy Ojalvo]

The thing is it compares the incidence in the same group of states from one time period to another. It doesn't compare different states to each other purely based off number. Hence I feel like I should still be able to make some meaningful findings using the percentage it decreased or increased from the previous period to the one after. Unless you think the state variance is still an issue, because a state like Texas having a decrease in lawsuits might play too much of an influence to the whole block.

[Clyde Schechter]

In light of your lasts two posts, I realize that I have not paid sufficient attention to the data structure you are working with. I had been assuming that you had one observation for each state in the pre-policy period and one in the post-policy state, those observations containing outcome variables like average malpractice payment. I should have realized that was impossible when I saw your outputs with sample sizes in the tens of thousands! My apologies for not pointing this out sooner and reducing the amount of re-work you now face.

You can't use -regress- to analyze this kind of data. You clearly have many observations in each state within both the pre- and post- eras. These observations are necessarily not independent. In addition, litigation settlements will be higher in high-cost-of-living states than in low-cost-of-living ones. -regress- does not take that into account either. So you need to use an analysis that does. The -xt- suite of commands is made for that. If you are not familiar with -xtreg- and similar commands, I suggest you read the manual sections on -xtset- and -xtreg-. Among the key conceptual points: the -xt- models when run with the -fe- option provide pure within-state estimations, so that systematic differences among the states are no longer a source of bias. This also reduces the correlation of observations within states. The -vce(cluster state)- option further adjusts the standard errors, confidence intervals, and p-values for the degree of correlation within states that remains.

I'll assume state is a numeric variable. If it's a string showing state names or abbreviations, then -encode state, gen(n_state)- will give you a numeric variable that you can read just as easily. Then you need to run -xtset state- (or -xtset n_state- if state is a string; -xtset- will not accept a string variable.)

Then re-run all of the -regress- analyses using -xtreg- instead. I recommend also specifying the -vce(cluster state)- (or n_state if state is string) and -fe-options. The -margins- commands can then be run as before, except that you must specify the -noestimcheck- option.

Your results may come out rather different from what you have seen. But these new analyses are far more correct.

For the number of litigations, I would reduce the structure of the data set to one observation per state in each of the pre- and post- eras.

collapse (count) n_lawsuits = payment2 (first)  apology, by(state pre_post)
xtset state
xtpoisson n_lawsuits i.apology##i.pre_post, fe vce(cluster state)
margins apology#pre_post, noestimcheck
margins apology, dydx(pre_post) noestimcheck

[Issy Ojalvo]

Ohh makes sense. I do have a numeric variable for the states called workstat2. I tried doing a few different things and I came up with errors. First I just did -xtset- workstat2 and ran

xtreg payment2 i.apology##i.pre_post, fe vce(cluster state)

but it gave me ""note: 1.apology omitted because of collinearity." I got results, but data is probably missing. Second I tried the code you just sent me to measure number of lawsuits and it told me that "option cluster is not allowed" for some reason when trying the Poisson model.

Just to clarify so we are on the same page I am looking at the effectiveness of a lawsuit on either payment costs or number of lawsuits by comparing states that have the certain law from 1-3 years before implementation to 3-5 years after implementation. I didn't purposely cluster states together, but they should be naturally clustered by the sake of them sharing the same type of law.

[Clyde Schechter]

OK. The "note: 1.apology omitted because of collinearity" is not an error and is not a problem. In writing #11, I edited it several times before finally posting. I had originally mentioned that you would encounter this warning message and not to be concerned about it. But somehow in the final edits I left it out. This situation arises because, by the definition of a DID design, apology will always be 0 or always be 1 within any state. Consequently it is colinear with the "fixed effect" for the state. This is expected, and raises no difficulties. It is not related to missing data; you would get this same message even if there were no missing data at all. The only complication it creates is the necessity of adding the -noestimcheck- option to the -margins- commands subsequently.

You are correct that -vce(cluster...)- is not supported witih -xtpoisson, fe-. I forgot about that. Use -vce(robust)- instead. In the context of -xtpoisson- they are equivalent.

I didn't purposely cluster states together, but they should be naturally clustered by the sake of them sharing the same type of law.

Not exactly. The clustering does arise naturally. But the issue is not that they share the same type of law. The issue is that there are many, many things that influence your outcome variables: the law you are studying is just one of them. For example, malpractice payments will be influenced by the cost of living in the state, and probably also by social factors that influence juries' sympathies. Those many things, other than the law, exert their influence in every observation in a given data set. As a result observations from the same state will tend to be more similar than observations from different states. These effects are the reason we treat the data as clustered within states.

[Issy Ojalvo]

These were the results I ended up getting. Some pretty interesting findings that "protfault"- or the apology law in which a doctor's claim that the incident is their own fault is inadmissible as evidence in court- actually led to a big increase in average malpractice payments. Now I just have to read up on stata/statistics so I can fully understand what I even calculated haha.

. xtreg payment2 i.apology##i.pre_post, fe vce(cluster workstat2)
note: 1.apology omitted because of collinearity

Fixed-effects (within) regression               Number of obs     =     62,701
Group variable: workstat2                       Number of groups  =         44

R-sq:                                           Obs per group:
     within  = 0.0005                                         min =         75
     between = 0.0109                                         avg =    1,425.0
     overall = 0.0032                                         max =      9,653

                                                F(2,43)           =       1.78
corr(u_i, Xb)  = 0.2016                         Prob > F          =     0.1809

                                 (Std. Err. adjusted for 44 clusters in workstat2)
----------------------------------------------------------------------------------
                 |               Robust
        payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
       1.apology |          0  (omitted)
      1.pre_post |   -5196.25   11769.09    -0.44   0.661    -28930.88    18538.38
                 |
apology#pre_post |
            1 1  |  -26623.25   20963.22    -1.27   0.211     -68899.6    15653.11
                 |
           _cons |   317403.7    5066.37    62.65   0.000     307186.4      327621
-----------------+----------------------------------------------------------------
         sigma_u |  90332.301
         sigma_e |  585491.59
             rho |  .02325028   (fraction of variance due to u_i)
----------------------------------------------------------------------------------

. margins apology#pre_post, noestimcheck

Adjusted predictions                            Number of obs     =     62,701
Model VCE    : Robust

Expression   : Linear prediction, predict()

----------------------------------------------------------------------------------
                 |            Delta-method
                 |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
apology#pre_post |
            0 0  |   317403.7    5066.37    62.65   0.000     307473.8    327333.6
            0 1  |   312207.5    11425.4    27.33   0.000     289814.1    334600.8
            1 0  |   317403.7    5066.37    62.65   0.000     307473.8    327333.6
            1 1  |   285584.2   12568.67    22.72   0.000     260950.1    310218.3
----------------------------------------------------------------------------------

. margins apology, dydx(pre_post) noestimcheck

Conditional marginal effects                    Number of obs     =     62,701
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
     apology |
          0  |   -5196.25   11769.09    -0.44   0.659    -28263.24    17870.74
          1  |   -31819.5   17347.77    -1.83   0.067     -65820.5    2181.501
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. xtreg payment2 i.genexp##i.pre_post, fe vce(cluster workstat2)
note: 1.genexp omitted because of collinearity
note: 2.genexp omitted because of collinearity

Fixed-effects (within) regression               Number of obs     =     62,701
Group variable: workstat2                       Number of groups  =         44

R-sq:                                           Obs per group:
     within  = 0.0007                                         min =         75
     between = 0.0594                                         avg =    1,425.0
     overall = 0.0053                                         max =      9,653

                                                F(3,43)           =       1.26
corr(u_i, Xb)  = 0.2796                         Prob > F          =     0.2989

                                (Std. Err. adjusted for 44 clusters in workstat2)
---------------------------------------------------------------------------------
                |               Robust
       payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------+----------------------------------------------------------------
         genexp |
             1  |          0  (omitted)
             2  |          0  (omitted)
                |
     1.pre_post |  -4541.811   12249.42    -0.37   0.713    -29245.12     20161.5
                |
genexp#pre_post |
           1 1  |  -41235.08   28872.65    -1.43   0.160    -99462.34    16992.17
           2 1  |  -4663.423   17158.73    -0.27   0.787    -39267.31    29940.46
                |
          _cons |   317271.1   4899.144    64.76   0.000     307391.1    327151.2
----------------+----------------------------------------------------------------
        sigma_u |  89418.111
        sigma_e |  585450.14
            rho |  .02279588   (fraction of variance due to u_i)
---------------------------------------------------------------------------------

. margins genexp#pre_post, noestimcheck

Adjusted predictions                            Number of obs     =     62,701
Model VCE    : Robust

Expression   : Linear prediction, predict()

---------------------------------------------------------------------------------
                |            Delta-method
                |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
----------------+----------------------------------------------------------------
genexp#pre_post |
           0 0  |   317271.1   4899.144    64.76   0.000       307669    326873.3
           0 1  |   312729.3   11766.09    26.58   0.000     289668.2    335790.4
           1 0  |   317271.1   4899.144    64.76   0.000       307669    326873.3
           1 1  |   271494.2   21752.82    12.48   0.000     228859.5      314129
           2 0  |   317271.1   4899.144    64.76   0.000       307669    326873.3
           2 1  |   308065.9   11669.46    26.40   0.000     285194.2    330937.6
---------------------------------------------------------------------------------

. margins genexp, dydx(pre_post) noestimcheck

Conditional marginal effects                    Number of obs     =     62,701
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
      genexp |
          0  |  -4541.811   12249.42    -0.37   0.711    -28550.23    19466.61
          1  |  -45776.89    26145.4    -1.75   0.080    -97020.93    5467.141
          2  |  -9205.234   12015.57    -0.77   0.444    -32755.32    14344.85
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. xtreg payment2 protfault##i.pre_post, fe vce(cluster workstat2)
note: 1.protfault omitted because of collinearity
note: 2.protfault omitted because of collinearity

Fixed-effects (within) regression               Number of obs     =     62,701
Group variable: workstat2                       Number of groups  =         44

R-sq:                                           Obs per group:
     within  = 0.0007                                         min =         75
     between = 0.0149                                         avg =    1,425.0
     overall = 0.0041                                         max =      9,653

                                                F(3,43)           =       2.05
corr(u_i, Xb)  = 0.2179                         Prob > F          =     0.1205

                                   (Std. Err. adjusted for 44 clusters in workstat2)
------------------------------------------------------------------------------------
                   |               Robust
          payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------------+----------------------------------------------------------------
         protfault |
                1  |          0  (omitted)
                2  |          0  (omitted)
                   |
        1.pre_post |   -5196.25   11769.18    -0.44   0.661    -28931.07    18538.57
                   |
protfault#pre_post |
              1 1  |   20143.85   16096.94     1.25   0.218    -12318.72    52606.41
              2 1  |   -34566.2   22864.42    -1.51   0.138     -80676.7    11544.29
                   |
             _cons |   317394.6    4925.08    64.44   0.000     307462.2      327327
-------------------+----------------------------------------------------------------
           sigma_u |  90072.887
           sigma_e |  585443.83
               rho |   .0231237   (fraction of variance due to u_i)
------------------------------------------------------------------------------------

. margins protfault#pre_post, noestimcheck

Adjusted predictions                            Number of obs     =     62,701
Model VCE    : Robust

Expression   : Linear prediction, predict()

------------------------------------------------------------------------------------
                   |            Delta-method
                   |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------------+----------------------------------------------------------------
protfault#pre_post |
              0 0  |   317394.6    4925.08    64.44   0.000     307741.6    327047.6
              0 1  |   312198.4   11363.52    27.47   0.000     289926.3    334470.5
              1 0  |   317394.6    4925.08    64.44   0.000     307741.6    327047.6
              1 1  |   332342.2   11619.06    28.60   0.000     309569.3    355115.1
              2 0  |   317394.6    4925.08    64.44   0.000     307741.6    327047.6
              2 1  |   277632.2   14986.04    18.53   0.000     248260.1    307004.2
------------------------------------------------------------------------------------

. margins protfault, dydx(pre_post) noestimcheck

Conditional marginal effects                    Number of obs     =     62,701
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
   protfault |
          0  |   -5196.25   11769.18    -0.44   0.659    -28263.43    17870.92
          1  |    14947.6    10981.7     1.36   0.173    -6576.135    36471.33
          2  |  -39762.45   19602.76    -2.03   0.043    -78183.15   -1341.758
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. xtreg payment2 disclos##i.pre_post, fe vce(cluster workstat2)
note: 1.disclos omitted because of collinearity
note: 2.disclos omitted because of collinearity

Fixed-effects (within) regression               Number of obs     =     62,701
Group variable: workstat2                       Number of groups  =         44

R-sq:                                           Obs per group:
     within  = 0.0004                                         min =         75
     between = 0.0061                                         avg =    1,425.0
     overall = 0.0002                                         max =      9,653

                                                F(3,43)           =       1.33
corr(u_i, Xb)  = -0.0414                        Prob > F          =     0.2780

                                 (Std. Err. adjusted for 44 clusters in workstat2)
----------------------------------------------------------------------------------
                 |               Robust
        payment2 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
         disclos |
              1  |          0  (omitted)
              2  |          0  (omitted)
                 |
      1.pre_post |  -30832.13   21412.51    -1.44   0.157    -74014.57     12350.3
                 |
disclos#pre_post |
            1 1  |   6007.564   34260.45     0.18   0.862    -63085.22    75100.34
            2 1  |   19081.35   24299.68     0.79   0.437    -29923.64    68086.33
                 |
           _cons |   317270.7   4955.803    64.02   0.000     307276.4      327265
-----------------+----------------------------------------------------------------
         sigma_u |  91087.802
         sigma_e |  585514.57
             rho |  .02362977   (fraction of variance due to u_i)
----------------------------------------------------------------------------------

. margins disclos#pre_post, noestimcheck

Adjusted predictions                            Number of obs     =     62,701
Model VCE    : Robust

Expression   : Linear prediction, predict()

----------------------------------------------------------------------------------
                 |            Delta-method
                 |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
disclos#pre_post |
            0 0  |   317270.7   4955.803    64.02   0.000     307557.5    326983.9
            0 1  |   286438.6   18280.85    15.67   0.000     250608.8    322268.4
            1 0  |   317270.7   4955.803    64.02   0.000     307557.5    326983.9
            1 1  |   292446.1   23773.22    12.30   0.000     245851.5    339040.8
            2 0  |   317270.7   4955.803    64.02   0.000     307557.5    326983.9
            2 1  |   305519.9   11207.58    27.26   0.000     283553.5    327486.4
----------------------------------------------------------------------------------

. margins disclos, dydx(pre_post) noestimcheck

Conditional marginal effects                    Number of obs     =     62,701
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
     disclos |
          0  |  -30832.13   21412.51    -1.44   0.150    -72799.87    11135.61
          1  |  -24824.57   26744.77    -0.93   0.353    -77243.36    27594.23
          2  |  -11750.78   11488.22    -1.02   0.306    -34267.28    10765.71
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

[Issy Ojalvo]

With the n_lawsuits, I'm not sure if it is giving me the right numbers or maybe I'm just misinterpreting the data. In the variable n_lawsuits, that you created the numbers do appear right. It's more so with the modeling. Also how would I go about doing the same modeling for the genexp, protfault, and disclos variables?

. xtset workstat2
       panel variable:  workstat2 (unbalanced)

. xtpoisson n_lawsuits i.apology##i.pre_post, fe vce(robust)

Iteration 0:   log pseudolikelihood =  -1548.224  
Iteration 1:   log pseudolikelihood = -319.24879  
Iteration 2:   log pseudolikelihood = -319.02494  
Iteration 3:   log pseudolikelihood = -319.02494  

Conditional fixed-effects Poisson regression    Number of obs     =         88
Group variable: workstat2                       Number of groups  =         44

                                                Obs per group:
                                                              min =          2
                                                              avg =        2.0
                                                              max =          2

                                                Wald chi2(2)      =     352.79
Log pseudolikelihood  = -319.02494              Prob > chi2       =     0.0000

                                  (Std. Err. adjusted for clustering on workstat2)
----------------------------------------------------------------------------------
                 |               Robust
      n_lawsuits |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
       1.apology |          0  (omitted)
      1.pre_post |  -.4459613   .0297531   -14.99   0.000    -.5042763   -.3876463
                 |
apology#pre_post |
            1 1  |   .0684329    .044695     1.53   0.126    -.0191677    .1560336
----------------------------------------------------------------------------------

. margins apology#pre_post, noestimcheck

Adjusted predictions                            Number of obs     =         88
Model VCE    : Robust

Expression   : Linear prediction, predict()

----------------------------------------------------------------------------------
                 |            Delta-method
                 |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
apology#pre_post |
            0 0  |          0  (omitted)
            0 1  |  -.4459613   .0297531   -14.99   0.000    -.5042763   -.3876463
            1 0  |          0  (omitted)
            1 1  |  -.3775284   .0333527   -11.32   0.000    -.4428984   -.3121584
----------------------------------------------------------------------------------

. margins apology, dydx(pre_post) noestimcheck

Conditional marginal effects                    Number of obs     =         88
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.pre_post

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.pre_post   |
     apology |
          0  |  -.4459613   .0297531   -14.99   0.000    -.5042763   -.3876463
          1  |  -.3775284   .0333527   -11.32   0.000    -.4428984   -.3121584
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.