Simulation failure with own program

[Dea Tusha]

Dear Statalist members,

I am using Stata 14.2 on Windows 7. I am running a simulation of a program I wrote. I am using gsample to draw (with repetition) samples from a panel dataset, calculating some variables based on the newly drawn sample, and then running xtreg on the sample. I then use simulate to repeat this up to 500 times and save the estimated beta coefficients and standard errors from the regressions.

Firstly, some of the variables in my fe regressions fall out due to collinearity and Stata shows the warning "captured error in x, posting missing value", but proceeds with the simulation. The first draw always comes up as a dot, but the rest are red x-s. This is not a problem per se, as I am not interested in the coefficients that cannot be estimated. However, when looking at my results, after a couple of "successful runs", the rest of the simulations all yield the same results (i.e. in the new dataset with all the estimated coefficients, all observations after the first couple of observations are the same -see attached picture below). I've run the simulation many times now, changing the seed everytime and have noticed that: 1. the number of "successful runs" varies per simulation and 2. the values that the estimates keep reverting to are the same even in different simulations, with different seed numbers.

Does anyone have any idea why this could happen and how to solve it? I should mention that running the program by itself several times seems to work fine and produces different estimates every time. I am attaching my code below:

capture program drop mypro
program define mypro, rclass
preserve
gsample nsample, strata(strata) cluster(firmid) idcluster(sampleid)
xtset sampleid year

bysort sameisic2r year: egen sindcy=total(consoutput)

*2.Market share & squared, Herfindahl index
gen smshare=consoutput/sindcy
gen smsharesq=smshare^2
by sameisic2r year: egen sherfi=total(smsharesq)

*3.Foreign output
bysort sameisic2r year: egen sforcy=total(consoutput) if for20==1
xtset sampleid year
egen sampleyr=group(sameisic2r year)
xtset sampleyr
xfill sforcy, i(sampleyr)
xtset sampleid year

*4.Horizontal FDI
gen shorfdi=sforcy/sindcy
gen skot=0
gen sbfdi=.
levelsof year, local(years)
foreach x of local years{
levelsof sameisic2r, local(sectors)
foreach y of local sectors {
gen shorfdi`x'`y'=shorfdi if year==`x'& sameisic2r==`y'
xtset skot
xfill shorfdi`x'`y', i(skot)
xtset sampleid year
gen wshorfdi`x'`y'=to`y'*shorfdi`x'`y'
egen sbackfdi`x'`y'=rowtotal(wshorfdi`x'*)
gen scbackfdi`x'`y'=sbackfdi`x'`y'- to`y'*shorfdi`x'`y' if year==`x'& sameisic2r==`y'
replace sbfdi=scbackfdi`x'`y' if year==`x'& sameisic2r==`y'
}
}

drop shorfdi2* wshorfdi2* sbackfdi2* scbackfdi2*

xtset sampleid year

gen sampletfp=.
foreach x of local sectors{
prodest logy if sameisic2r==`x', free(logl) proxy(logm) state(logk) method(lp) control(loge) acf translog
predict sampletfp`x' if sameisic2r==`x', resid
replace sampletfp=sampletfp`x' if sameisic2r==`x'
}

xtreg sampletfp sbfdi shorfdi sherfi i.year year#dprovi if foreign==0, fe cluster(sameisic2r)

restore
end

simulate _b _se, seed(1510) reps(500) saving(try1, double replace): mypro




Kind regards,
Dea Tusha

[Dea Tusha]

UPDATE: I believe I may have found a solution to my problem and am sharing it here in case someone else comes across the same issue. After doing some more searching online, I came across this thread: https://www.statalist.org/forums/forum/general-stata-discussion/general/1302259-bootstrap-and-monte-carlo, which was dealing with a similar problem. The suggestion there by Maarten Buis was that the problem arose because there was a bootstrap within the simulation, and a different seed was set for the bootstrap.

In my case, I am making use of the prodest command in the simulation, which in itself uses a bootstrap to produce standard errors. The solution was to change the .ado file for the prodest command, so that it stops setting a seed by default, and only setting a seed in my simulation command, as suggested by Maarten in the thread above. This seems to have solved the issue (I only ran a simulation with 20 replications for now, but I expect that this will still be the case once I run all 500 replications).

[Clyde Schechter]

This solution makes sense. Thank you for closing the thread and making the knowledge available to others.