restore command suddenly stopped working.

[Anselm Schulz]

Hello!
I ran into a problem after modifying a .do file from a previous analysis.
When using the restore command after earler using preserve I get an error "nothing to restore".

Stata version 18. I am using the ssc command diagt but that was working fine before modifications.

Sorry that the code is not executable due to many locals - I was hoping that anyone could spot the issue right away.
If that is difficult I will try to provide executable code.

Here how the code looked in the beginning and everything works fine:

preserve

keep if any_lesion == 1 & dp_study == 0

local description All_Lesions_Pilot

local counter = 2

foreach var in `varlist_detect' {
        local description "`description' `var' "
        
        
        diagt dp_RS_lesion_detect `var'
        
        replace description = "`var'" in `counter'
        replace sensitivity = `r(sens)' in `counter'
        replace sens_lb = `r(sens_lb)' in `counter'
        replace sens_ub = `r(sens_ub)' in `counter'
        replace PPV = `r(ppv)' in `counter'
        replace PPV_lb = `r(ppv_lb)' in `counter'
        replace PPV_ub = `r(ppv_ub)' in `counter'
        
        *McNemars
        mcc tc_dp_m_as_df_lesion_detect tc_`var'
        replace p_value = `r(p_exact)' in `counter'
        
        local ++counter
}

* Store to matrix to prevent deløetion by "restore"
mkmat `varlist_sensitivity', matrix(pilot_all)

local n = _N

levelsof RS_patient
local p_n = `r(r)'

restore

* Matrix to variables after restore

local counter = 1
local counter_store = (`counter'+1)

foreach i in `description' {
    replace description = "`i'" in `counter'
    local ++counter
}


forvalues i = 2/7 {
    replace sensitivity = pilot_all[`i',1] in `counter_store'
    replace sens_lb = pilot_all[`i',2] in `counter_store'
    replace sens_ub = pilot_all[`i',3] in `counter_store'
    replace PPV = pilot_all[`i',4] in `counter_store'
    replace PPV_lb = pilot_all[`i',5] in `counter_store'
    replace PPV_ub = pilot_all[`i',6] in `counter_store'
    replace lesions_n = `n' in `counter_store'
    replace patients_n = `p_n' in `counter_store'
    replace p_value = pilot_all[`i',7] in `counter_store'
    local ++counter_store
}

and after modifications the restore command does not work anymore:

preserve


keep if RS_lesion_detect != . & dp_study == 0

local description "`reader': All Lesions Pilot"

local counter = 2

set trace on


foreach reader in `reader' {
    
unab varlist_sensitivity: `reader'_sens* `reader'_PPV* `reader'_p_value

foreach var in ``reader'_varlist_detect' {
        local description "`description' `var' "
        
        
        diagt binary_RS_lesion_detect `var'
        
        replace `reader'_description = "`var'" in `counter'
        replace `reader'_sensitivity = `r(sens)' in `counter'
        replace `reader'_sens_lb = `r(sens_lb)' in `counter'
        replace `reader'_sens_ub = `r(sens_ub)' in `counter'
        replace `reader'_PPV = `r(ppv)' in `counter'
        replace `reader'_PPV_lb = `r(ppv_lb)' in `counter'
        replace `reader'_PPV_ub = `r(ppv_ub)' in `counter'
        
        *McNemars
        local mcc_var = subinstr("`var'", "m_","",.)
        local mcc_var = subinstr("`mcc_var'", "binary_","",.)
        
        mcc tc_`reader'_as_df_lesion_detect tc_`mcc_var'
        replace `reader'_p_value = `r(p_exact)' in `counter'
        
        local ++counter
        
}

* Store to matrix to prevent deletion by "restore"
mkmat `varlist_sensitivity', matrix(pilot_all)

local n = _N

levelsof RS_patient
local p_n = `r(r)'

restore


* Matrix to variables after restore

local counter = 1
local counter_store = (`counter'+1)

foreach i in `description' {
    replace `reader'_description = "`i'" in `counter'
    local ++counter
}


forvalues i = 2/7 {
    replace `reader'_sensitivity = pilot_all[`i',1] in `counter_store'
    replace `reader'_sens_lb = pilot_all[`i',2] in `counter_store'
    replace `reader'_sens_ub = pilot_all[`i',3] in `counter_store'
    replace `reader'_PPV = pilot_all[`i',4] in `counter_store'
    replace `reader'_PPV_lb = pilot_all[`i',5] in `counter_store'
    replace `reader'_PPV_ub = pilot_all[`i',6] in `counter_store'
    replace `reader'_lesions_n = `n' in `counter_store'
    replace `reader'_patients_n = `p_n' in `counter_store'
    replace `reader'_p_value = pilot_all[`i',7] in `counter_store'
    local ++counter_store
}


}

Thanks for any suggestions and support!!

[Leonardo Guizzetti]

There’s isn’t a reproducible example here, but preserve and restore are, like many commands, are sensitive to the context of executing interactively versus via a do file. For example, you can’t preserve data when running the do file then later, use restore interactively (typing into Stata’s command prompt). This is when you might get that error and I’d expect is the source of your problem.

[Anselm Schulz]

Sorry! I found the error - it is due to the new loop and occurs after the second run. Uffff... sorry!

[Minna Tuominen]

Hello, I am also using Stata18, and having a similar problem with the restore-command:


I am working with pooled PISA-data from survey waves 2009-2022, but would like to generate and save a separate data-file using information on specific years only. I am using the following code:

preserve // Originally 756,540 observations

keep if wave == 2009 | wave == 2012 // Nr. of observations drops to 320,042

repest PISA, estimate (means pv@math pv@read pv@scie) over(wave) by(gender) outfile(means0912)

restore

Stata executes the commands as expected, but when hitting the restore-command, it says "nothing to restore".

I thought the user-written repest-command (by Francesco Avvisati & Francois Keslair) might be the reason for the problem but the situation remains the same, i.e. the restore-command doesn't work, even if I run a standard command, such as OLS-regression. Below, is a naïve set of commands that I run just to test the restore-function:

clear all

use PISA_all_waves_mod.dta

preserve // Originally 756,540 observations

keep if wave == 2009 | wave == 2012 // Nr. of observations drops to 320,042

reg pv1math female age misced fisced
est sto OLSmath

reg pv1read female age misced fisced
est sto OLSread

reg pv1scie female age misced fisced
est sto OLSscie

esttab OLS* using olsmodels.rtf, replace

restore

Stata declares: "Nothing to restore"


Can you please, advise what I should do differently?

[Leonardo Guizzetti]

You don't say precisely how you executed those commands, but there is nothing I see that would cause it to not work. You would get the same error had you erroneously tried to use -restore- twice, where the second time there really is nothing to restore as it had already been done.

-preserve- and -restore- are context dependent, meaning that they only work for the currently active execution. If you run the entire program line by line in the command window (interactive mode), or the entire program using Ctrl+D, each will work. But if you had run part of the command using Ctrl+D to run the lines and then run the rest interactively, this would not work because they don't have the same context. Try to rerun the commands running entire do file at once and see if you still get the same error.

[Minna Tuominen]

Hi Leonardo,

many thanks for your advice! It solved the problem! :-)

Initially, I ran the commands line by line from do-file (sorry for not being precise in my first message), as I wanted to follow the changes in the dataset. When reaching the restore-line, Stata would say "Nothing to restore". Now, I reran the commands, as you said, still using do-file but highlighting and executing the commands from preserve to restore, all at once, and to my surprise, the original dataset with 756,540 observations was fully restored! Quite incredible! I have never run into a similar problem with other commands, and I think this isn't quite how the commands should work. Anyway, now I know how to deal with preserve and restore. Many thanks to you!