Set seed command not respected for -bsqreg-

[Philip Jones]

Hi all,

I am using the following code to attempt to get reproducible results for a median regression I am doing.

set seed 339487731
bsqreg minFPR ib6.journal ib3.intervention ib7.fundingsource ib2.trialcenter ib2.location ib3.outcome c.ss_10, cformat(%9.2f) reps(1000)

But, after quitting Stata and running the do file again, my standard errors change slightly.

Am I missing something? Does anyone have any suggestions?

Many thanks.

[William Lisowski]

The code below was an attempt to create a reproducible example, but it failed - or conversely, succeeded in obtaining the same results with the same seed.

. sysuse auto, clear
(1978 automobile data)

. 
. * #1 set seed
. set seed 339487731

. bsqreg price weight length foreign
(fitting base model)

Bootstrap replications (20)
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5 
....................

Median regression, bootstrap(20) SEs                Number of obs =         74
  Raw sum of deviations  71102.5 (about 4934)
  Min sum of deviations 54411.29                    Pseudo R2     =     0.2347

------------------------------------------------------------------------------
       price | Coefficient  Std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
      weight |   3.933588   2.838886     1.39   0.170    -1.728392    9.595567
      length |  -41.25191   75.37362    -0.55   0.586    -191.5799     109.076
     foreign |   3377.771   1315.596     2.57   0.012      753.897    6001.645
       _cons |   344.6489   6487.153     0.05   0.958    -12593.57    13282.87
------------------------------------------------------------------------------

. 
. * #2 run again without resetting seed - different results
. bsqreg price weight length foreign
(fitting base model)

Bootstrap replications (20)
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5 
....................

Median regression, bootstrap(20) SEs                Number of obs =         74
  Raw sum of deviations  71102.5 (about 4934)
  Min sum of deviations 54411.29                    Pseudo R2     =     0.2347

------------------------------------------------------------------------------
       price | Coefficient  Std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
      weight |   3.933588   2.619589     1.50   0.138    -1.291019    9.158194
      length |  -41.25191   67.75221    -0.61   0.545    -176.3794     93.8756
     foreign |   3377.771   1175.075     2.87   0.005     1034.158    5721.384
       _cons |   344.6489    6169.65     0.06   0.956    -11960.33    12649.63
------------------------------------------------------------------------------

. 
. * #3 set same seed as #1 - same results as #1
. set seed 339487731

. bsqreg price weight length foreign
(fitting base model)

Bootstrap replications (20)
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5 
....................

Median regression, bootstrap(20) SEs                Number of obs =         74
  Raw sum of deviations  71102.5 (about 4934)
  Min sum of deviations 54411.29                    Pseudo R2     =     0.2347

------------------------------------------------------------------------------
       price | Coefficient  Std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
      weight |   3.933588   2.838886     1.39   0.170    -1.728392    9.595567
      length |  -41.25191   75.37362    -0.55   0.586    -191.5799     109.076
     foreign |   3377.771   1315.596     2.57   0.012      753.897    6001.645
       _cons |   344.6489   6487.153     0.05   0.958    -12593.57    13282.87
------------------------------------------------------------------------------

This was run on Stata 17.0.

Are you sure that the data was sorted in the same order for the two runs? Can you rerun your test changing it to use your input dataset immediately before the set seed and bsqreg commands?

[Philip Jones]

William,

Thank you for the rapid reply.

You were absolutely correct. I failed to appreciate the sensitivity of the sort order to the reproducibility.

My "problematic" command was part of a longer do file, and, by executing portions of it, I inadvertently changed (or did not change, depending) the sort order with the commands run (or not run).

Once I sorted the dataset just before the command, all worked as it should have.

Thank you again for stopping me from banging my head against the wall! 🙂