Difference-in-difference regression interpretation

[Will Murphy]

Good afternoon,

I know there have been previous posts regarding this topic, however I first wanted to verify what I have done is a simple difference-in-difference estimation (my knowledge regarding Stata and econometrics is limited) and then ask, even if not a DID, how to interpret the coefficients shown.

To note: 'logGRSSWK' = log of gross weekly wage
'DIS' =1 if disabled and = 0 if non-disabled
: 'ACT'=1 if after the implementation of a legislation and 0= if before.

 reg logGRSSWK ACT DIS ACTDIS i.SEX i.ETH i.AGES i.RES_NEW i.REGWKR_NEW i.HDPCH19 i.IND i.MARSTA i.HIQUAL i.FLOW i.SKSBN91 i.FTPTWK, robust

Linear regression                               Number of obs     =      5,039
                                                F(80, 4958)       =      83.57
                                                Prob > F          =     0.0000
                                                R-squared         =     0.5940
                                                Root MSE          =     .54172

-------------------------------------------------------------------------------------------------------------------------
                                                        |               Robust
                                              logGRSSWK |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
--------------------------------------------------------+----------------------------------------------------------------
                                                    ACT |   .1177817   .0179667     6.56   0.000      .082559    .1530045
                                                    DIS |  -.0600279   .0285709    -2.10   0.036    -.1160396   -.0040162
                                                 ACTDIS |  -.0053434   .0405925    -0.13   0.895    -.0849226    .0742357
                                                        |

Any help would be greatly appreciated, thank you.

[Clyde Schechter]

You don't say what ACTDIS is, but I'll assume that it's calculated as ACT*DIS. If that is true, then, yes, this is a DID model.

The main conclusion is that the DID estimate of the causal effect of ACT on logGRSSWK is a reduction of 0.0054 (95% CI -0.085 to _0.074). We note that among the non-disabled, logGRSSWK was 0.1178 higher after the implementation of ACT than before, and that the pre-implementation mean logGRSSWK was 0.06 lower among the disabled than among the non-disabled.

If you prefer to interpret the findings in the non-log transformed GRSSWK metric, the implementation of the legislation reduced GRSSWK by a factor of about 0.0053 (= 1 - exp(-0.0053434)), or by 0.53%.

[Will Murphy]

Thank you so much for your help- apologies, you were correct, ACTDIS is ACT*DIS.

If I may ask 2 follow-ups, as this is a rather preliminary step I undertake in assessing the impact of the legislation on the disabled within the workplace:

- Firstly, this DID is very limited as the dataset composition changes between pre- and post- act, because this allows for people switching between employment statuses and, using a logged dependent variable when wage could be 0 for unemployed, means the composition is uneven. Between 2 alternatives- 1 being using the non-log transformed GRSSWK metric, and the other being maintaining an even composition via only assessing those in employment pre- and post- act (the latter limiting my ability to truly assess disability in the workplace)- which is less econometrically flawed in your opinion?

- Secondly, once adjusting this first issue I have the issue of endogeneity, caused by this legislation expanding the definition of disability (i.e there are more disabled people after ACT). A particular concern is people may, due to better employment opportunities imposed by this legislation, opportunise and classify as disabled (regardless of whether they are or not). If I was to create dummy variables for disability before and after ACT and then compare the relative impacts on (log) GRSSWK of changing disability status pre- and post- ACT, could this suffice as an attempt to address and assess a possible endogeneity concern?

Any advice/ help would be appreciated and thank you again for the reply.

[Clyde Schechter]

Regarding your first question, I am not an economist, so I cannot advise you on what is econometrically sound. That said, I don't see how using a non-log-transformed metric deals with this issue anyway. (That's not to say I'm opposed to your using the non-transformed GRSSWK, just that I don't see it as a solution to this particular problem. I think you need to evaluate whether the log-transformed or non-log-transformed gives a better fitting model and go with that.) The other approach of selecting only those who were employed both pre- and post- clearly introduces a selection bias. Not only does it limit the generalizability of your results, it actually conditions them on something, future continued employment, that cannot be identified prospectively--so you can never know for any person whether your results would be applicable to them or not. I think economists have ways of dealing with this kind of problem involving simultaneous estimation of both the GRSSWK outcome and employment status,, the name Heckman comes to mind, but I don't know much about these and I can't advise you.

The second question is really thorny. If you have the information needed to impose the pre-intervention definition of disability on all observations and use that as your disability variable, that would overcome at least part of the problem. But if the self-identification of disability/not-disability under the new law also affected decisions about whether to work, or affected job mobility, then you still have a problem. In my line of work, epidemiology, we don't often encounter this kind of difficulty, so, again, I can't advise you beyond this most superficial level.

I recommend you seek further guidance from an economist or econometrician who is experienced in dealing with this kind of data. There are several who follow the Forum, and perhaps one of them will see this thread and chime in. If nobody does within a few days, you might consider posting these questions as a new thread with a title that will attracat the attention of those people.

[Will Murphy]

Thank you very much for your help Clyde- I will look further into the Heckman selection model and assess the problem at hand regarding the second question. Kind regards.

[Will Murphy]

I've just checked regarding the second question- I had created the disability dummy variable from 'DISEXT' (the extent of disability):

  tab DISEXT

                                 DISEXT |      Freq.     Percent        Cum.
----------------------------------------+-----------------------------------
DDA disabled and work-limiting disabled |        279        6.94        6.94
                           DDA disabled |        309        7.68       14.62
            Work-limiting disabled only |        147        3.65       18.27
                           Not disabled |      3,287       81.73      100.00
----------------------------------------+-----------------------------------
                                  Total |      4,022      100.00

                

where DDA disabled is the pre-intervention of disability. So, just to clarify, if I was to create a pre-intervention disability dummy from 'DDA disabled and work- limiting disabled' and 'DDA disabled', would I then compare it to the impact presented in my first post of this thread?
(To note: the frequency of DISEXT contrasts DIS in my first post due to current alterations made, for which I would ensure did not exist when comparing).

[Will Murphy]

Sorry for the confusion: my last post was regarding having pre-intervention definition of disability, however I mis-read it and assumed you meant regarding previous disability legislation. Would this be more ideal to overcome part of the problem as opposed to having pre-intervention definition of disability as simply those who recognised as disabled prior to the intervention vs those who do after?

[Clyde Schechter]

I'm not sure I understand what you are saying in #6 and #7. But if DDA disability is the definition of disability that prevailed prior to the intervention you are studying, then, yes, using a variable that combined DDA disabled and work-limiting disabled with DDA disabled as the "disabled" category and the others as "not disabled," this would go a long way towards solving your problems. It would eliminate the endogeneity of the "disabled/not disabled" distinction altogether.

[Will Murphy]

Thank you so much. Unfortunately the proportions under 'DDA disabled and work-limiting disabled' and 'DDA disabled' both increase after the intervention, so therefore this cannot be done and most likely have to delve into profit models and transition matrices to further address disability/non-disability endogeneity concerns.

[Will Murphy]

Sorry, an afterthought I had (and apologies if this sounds somewhat repetitive or basic) regarded the possibility of, after adopting the Heckman selection model, I was to create dummy variables for: disability status staying the same pre- and post-; not disabled to disabled; disabled to non-disabled and compared the relative wage impacts (like the DID in #1). In your opinion, would this be a somewhat effective method in reducing this possible source of endogeneity? Many thanks.

[Clyde Schechter]

As I indicated previously, we are veering into territory beyond what I really know well enough to advise you. I think the answer to your question in #10 is yes, but I do not feel sufficiently confident to advise you on it. Let me reiterate my advice in the final paragraph of #4: repost this question in a new thread with a title that will attract the attention of an economist. (Include a link to this thread in that new post.) I think you will get better, or at least more informed, advice that way.

[Will Murphy]

Ok that's perfect. Thank you for your advice and all your help.