how to know whether a placebo test is passed

[Yun Cheng]

Hello!

I'm conducting a placebo test on a staggered DID, I randomly assign treatment firms and policy year to my sample firms, repeat that 500 times, and I obtain the following graph. I read some related literature, however, they seem to be vague about what is the threshold of passing/not passing a placebo test. I wonder whether the graph below could be accounted as passing the placebo test, and why or why not so. Thanks for any advice!

[George Ford]

Looks more like Randomized Inference. If the true coefficient is within the CI of the randomized effects, then not significant. Can also randomize to get the boundaries of the t-stat, which can then be compared to the t-stat.

[Yun Cheng]

Hi again,

Just following up on this question, most of the p values in the graph in #1 are 0, which is significant. So does that mean the placebo test is not passed, as other fake events in fake event time can also generate significant results, just that the magnitude for most of them is not as small as the real estimate. Any ideas about this? Thanks!

[Joro Kolev]

Hi again,

Just following up on this question, most of the p values in the graph in #1 are 0, which is significant. So does that mean the placebo test is not passed, as other fake events in fake event time can also generate significant results, just that the magnitude for most of them is not as small as the real estimate. Any ideas about this? Thanks!

I think I already gave you my opinion on another thread. But in case I have not, I think this graph shows that the placebo test is not passed at standard levels (like 5% or 10%).

What matters, I think, is whether the coefficient with the actual data is unusual in the set of coefficients estimates with randomly reshuffled data. Your coefficient, which I presume is the red vertical line, is somewhat in the left tail, but it seems not extreme enough to be considered unusual under the reshuffled data estimates.

You might want to compute what are under the density the red line cuts off to the left.

[Yun Cheng]

Hi Joro,

Thanks for your help and sorry for the really late reply.

You might want to compute what are under the density the red line cuts off to the left.

Do you know how to achieve this technically? What is the code for computation? Thanks!