A cautionary tale

[Felix Bittmann]

This article here got published: https://www.psypost.org/criminalizin...e-study-finds/ (Maybe the original paper at Springer is already retracted but you can find the WP here: https://mpra.ub.uni-muenchen.de/1003...per_100393.pdf)

As it turns out, the results are driven by mishandling Stata: https://www.uio.no/studier/emner/mat...rickne2024.pdf

Even if this is not my area of expertise, I found this interesting to learn. Maybe this is helpful for others as well.

[George Ford]

Nice work. You'd think the graphical results would have indicated a problem with the model, given the large coefficient. I bet Stata reported omitted coefficients, which also indicates a bad model. Maybe an RED error message should be added when coefficients can't be estimated--to signal of a potential problem.

Statalist has made me quite the skeptic of published works. And, has taught me much. The ordering of i.x terms is a something I picked up here. I post of lot of possible answers to the simpler questions, but I'm mostly here to learn.

[Andrew Musau]

River Huang and colleagues wrote a nice article in Energy Economics on this exact issue. In their article, they identify three different papers (#1 not included) that have made this same mistake, so this is more common than you'd think. I believe the problem lies in people running a regression and not paying attention to the output. StataNow 18.5 has an absorb() option in xtreg,fe, so it should be easier to identify collinearities by absorbing the time variable.

[George Ford]

As for the Energy Economics comment, I always aggregate my data to the highest level of aggregation to avoid bias.

[Daniel Schaefer]

Super interesting stuff. I saw the original article, but would probably have been unaware of the response if not for this forum.

[Daniel Schaefer]

I always aggregate my data to the highest level of aggregation to avoid bias.

Unfortunately, I cannot always do this because I often care about important variation that exists at the local level, which can sometimes average out at higher levels of aggregation, so this is all particularly relevant to my work.

[Jeff Wooldridge]

One definitely has to be careful. Another way this manifests itself is in two-way fixed effects with age or potential experience included. Sometimes you'll get coefficients on these variables with a full set of time dummies. The effects are not identified but might be estimated if Stata drops one of the time dummies.

[Nick Cox]

My reaction here is of different kind. A paper has been published

Ciacci, R. Banning the purchase of sex increases cases of rape: evidence from Sweden. J Popul Econ 37, 37 (2024). https://doi.org/10.1007/s00148-024-00984-2

and heavily criticised

https://www.uio.no/studier/emner/mat...rickne2024.pdf

I have no interest myself, let alone expertise, in this highly popular but specialised area and so no intention of reading the papers. I also have no relation to any person mentioned.

But in principle the first author surely has a right of reply.

If the paper is retracted I will know what to think, but otherwise we should please be a little more circumspect in our comments, especially as someone's reputation is in question.