[Clyde Schechter]

It's good that you caught that problem in your data.

There isn't really a straightforward way to compare these results. Certainly 1.genexp and 1.disclos have some nice looking main effects. Don't forget, though, that the confidence intervals are wide, and even small effects in the opposite direction remain consistent with your data, though the bulk of the confidence intervals lie in the negative (i.e. desirable) territory.

[Issy Ojalvo]

So I accidentally did something I might regret. I ended up playing around with the data out of curiosity to see how the states did from the 1-3 years before implementation of their specific apology laws to all states in 2013-2015.

I essentially just reused the code you have sent me, but tailored for pre_post2 in which the 1 = all states from 2013-2015.

xtset workstat2
xtreg payment2 i.apology##i.pre_post2, fe vce(cluster workstat2)
margins apology#pre_post2, noestimcheck
margins apology, dydx(pre_post2) noestimcheck
xtreg payment2 i.genexp##i.pre_post2, fe vce(cluster workstat2)
margins genexp#pre_post2, noestimcheck
margins genexp, dydx(pre_post2) noestimcheck
xtreg payment2 protfault##i.pre_post2, fe vce(cluster workstat2)
margins protfault#pre_post2, noestimcheck
margins protfault, dydx(pre_post2) noestimcheck
xtreg payment2 disclos##i.pre_post2, fe vce(cluster workstat2)
margins disclos#pre_post2, noestimcheck
margins disclos, dydx(pre_post2) noestimcheck

collapse (count) n_lawsuits = payment2 (first) apology genexp protfault disclos, by(workstat2 pre_post2)
xtset workstat2
xtpoisson n_lawsuits i.apology##i.pre_post2, fe vce(robust)
margins apology#pre_post2, predict(nu0) noestimcheck
margins apology, dydx(pre_post2) predict(nu0) noestimcheck
xtpoisson n_lawsuits i.genexp##i.pre_post2, fe vce(robust)
margins genexp#pre_post2, predict(nu0) noestimcheck
margins genexp, dydx(pre_post2) predict(nu0) noestimcheck
xtpoisson n_lawsuits i.protfault##i.pre_post2, fe vce(robust)
margins protfault#pre_post2, predict(nu0) noestimcheck
margins protfault, dydx(pre_post2) predict(nu0) noestimcheck
xtpoisson n_lawsuits i.disclos##i.pre_post2, fe vce(robust)
margins disclos#pre_post2, predict(nu0) noestimcheck
margins disclos, dydx(pre_post2) predict(nu0) noestimcheck

The reason why I might regret this is because using this method it would appear that states without any type of apology laws preformed better in decreasing their malpractice payments and number of lawsuits compared to the rest of the states. This would pretty much invalidate my argument, or at least relegate my argument to only describing the short-term effects of these laws. Do I have to change the code to get accurate results considering the time between before and after now varies between states? Whats your opinion on this whole matter? I was close to finishing and now I'm mad at myself haha.

[Clyde Schechter]

I'm sorry, but this thread has gotten very long and I don't remember what your original pre_post variable was, so it's hard to put this all into context. Can you remind me and explain what the difference between them is?

It might also be a good idea to review why you even considered two different definitions for pre_post. That is normally a rather cut-and-dried matter that is not subject to judgment or exploration.

[Issy Ojalvo]

Pretty much pre_post is just a dummy variable created to indicate the before and after implementation of each states law. It will be ==0 if the years are 1-3 years before implementation of their law, or 2001-2004 for the states without apology laws, and it will be ==1 the 3-5 years after implementation of a states apology law, or 2007-2009 if the state does not have an apology law. All I did was create a new variable that made the ==0 the same, but the ==1 is all states from 2013-2015. Pretty much every state has effectively decreased their number of lawsuits by 80-92% over the years, and average payments in states without apology laws are way decreased.

I saw that the number of lawsuits decreased significantly so I just wanted to check out how states are doing more recently. For some reason or another, malpractice lawsuits are nearly eliminated, with around 3-4 states only recording one in 2015. I just feel like I don't have an argument anymore unless I did the code wrong.

[Clyde Schechter]

Well, if all you did is redefine the pre-post variable and then reuse the same code as before, there doesn't seem to be much room for a coding error.

Taking your results at face value, it seems that there was a dramatic drop in the number of malpractice lawsuits between 2009 and 2013 and that this decrease has swamped whatever benefits the apology law may have created between 2007 and 2009. The Great Recession may have played a role here. You may be aware that health care spending in the US was unusual during the 2010-2013 era (the rate of growth was lower than before). This might suggest that fewer health care services were being provided, and hence fewer opportunities for bad outcomes that lead to malpractice suits. In addition, I know that during that period many health care providers added (sneaked, really) mandatory arbitration clauses into their boilerplate paperwork for patients--which would also lead to a decrease in malpractice lawsuits. On the other hand, what you describe is the near-extinction of the malpractice suit in certain states. That seems surprising to me. I wonder if there is a problem with data collection. Is there a reporting lag, so that the counts provided by your data source are incomplete for recent years?

[Issy Ojalvo]

According to the NPDB website "Reports must be submitted within 30 days of the date an action was taken or a medical malpractice payment was made." If lawsuits get dragged out over 1.5-2 years I can see there being a reporting lag.

I just checked out the new data online without downloading the whole data base and it would seem the Vermont and Wyoming, two states that only reported one lawsuit, ended up having 13 and 15 lawsuits (assuming the data is up to date) in 2015. I downloaded my current data set back in December 2016 so there has been a good amount of time for the database to update. There was a 38% total decrease in malpractice lawsuits from 2002-2015, so definitely not as crazy as the 80-90% decrease originally reported.

I might just say that I do not feel comfortable making any long-term predictions at a state level using the NPDB, but I could give some national statistics using their updated website database. Those factors you mentioned are certainly important to take into account!

[Issy Ojalvo]

Curious, do you think there's any other statistical numbers I should include when making an argument? I know the p value in this case doesn't make sense but perhaps one of the other numbers presented?

[Clyde Schechter]

The key numbers are the expected mean outcomes and the marginal effects and, of course, the DID estimate itself, along with their confidence intervals. Before presenting those results, you will want to also present descriptive statistics about the data corpus you worked with. How many states contributed data to it? Over what periods of time? Which states were in each group? How do the groups compare on dimensions like overall population, urban/rural, mean and median income, unemployment rates, hospitals and doctors to population ratios, lawyers to population ratios--things like that.

And as a part of the presentation of the key results, a picture may well be worth thousands of words: show the key findings graphically.

[Issy Ojalvo]

What do you mean by the expected mean outcomes and the marginal effects? I reported the DID estimate itself and the confidence intervals.

[Clyde Schechter]

Well, in a presentation I would also show the mean payments (and numbers of lawsuits if you're including that outcome as well) in both groups before and after the introduction of the apology laws, and I would show the change between before and after in each group. These are the things you got from -margins-. These things provide context for the DID estimate.

Think of yourself being in the audience where somebody is giving a talk about, say, the effect of a new treatment for obesity, and they said that the people in the treatment group lost 5 kg more than those in the control group over the course of the study, wouldn't you want to know

1. So how much weight was lost in each group? and 2. What was the average weight in each group before and after treatment?

So those are the marginal effects in each group and the expected means in each group before and after treatment.