A question on how to bootstrap placebo test 1,000 times

[Ke Gong]

Dear Stata Experts,

Hope all is well. I'm trying to do a placebo test with this purpose: Randomly assign a fake IV called "origin" to an existing dataset and run logit regression on "origin" immediately, then repeat this process 1,000 times, and lastly plot the histogram distribution of the fake origin effect. Here is what I'm doing:

gen origin= (runiform() <= 0.5 )
logit DV origin, vce(r)
bootstrap _b, reps(1000) saving(my_results_file, replace): logit DV origin , vce(r)
use my_results_file, clear
hist _bs_1

However, I feel like the above syntax only randomly generated the fake "origin" IV ONCE and then ran the logit regression repeatedly 1,000 times. That's not what I wanted in the Placebo Test. I wonder if anyone can help me with this. Thanks so much!

[Clyde Schechter]

Yes. The -bootstrap:- prefix in the third line applies only to that regress command. It does not "look back" to repeat earlier commands.

Moreover, this isn't a job for -bootstrap- in any case. You are not looking to resample (with replacement) the originally created data set, which is what bootstrapping does. You are looking to retain the original data set, but replace the values of origin with random 0/1 values each tie through. This is a job for -simulate-.

capture program drop one_iteration
program define one_iteration, eclass
    replace origin = runiform() < 0.5
    logit DV origin, vce(r)
    exit
end

gen origin = .
tempfile results
simulate, reps(1000) saving(`results') dots(10) seed(1234): one_iteration

use `results', clear

Notes: The -seed()- option sets the random number seed so that your results are reproducible. The seed can be any positive integer you like. The -dots(10)- option tells Stata to print a dot on the screen after every 10 repetitions. This way you can see that you are making progress. As this runs pretty quickly and you are doing 1,000 reps, I thought every 10 repetitions would show that you are moving forward rapidly and not drown your Results screen with a thousand dots.

By the way, you don't need to use a -tempfile- for storing your results. A regular file will also do.

[Ke Gong]

@Clyde Schechter
Dear Dr. Schechter,

Thank you so much for your quick and nice solution! It works pretty well! I do appreciate your help. Now I'm able to use -hist- to plot the histogram of the estimated coefficients. Is it possible that I can also plot their associated p-values in the same graph?

[Ke Gong]

@Clyde Schechter

Please allow me to ask a follow-up question: if I replace the Logit regression with a Poisson regression, the simulation freezes and wouldn't respond during the simulate step. Should I revise the syntax for the Poisson regression? Thanks again!

[Clyde Schechter]

In order to troubleshoot, I need to see example data that reproduces the problem and also the exact code you are using.

[Ke Gong]

Dr. Schechter, thank you very much. I figured it out, the i.region I used caused the non-convergence. After removing it, the issue is fixed.