Within .ado files, is there a way to restore data but keep, until the end of program, tempvars created by the program?

[Danny Lempert]

Dear All,
I have a somewhat involved problem, but I think it boils down to this. I'm writing a program where I basically want to, in order:

1) modify the user's data
2) create temporary variables
3) restore the user's data
4) do some other stuff that requires the data as restored in 3)
5) use the temporary variables created in 2)


The problem with using preserve before step 1) and restore to do step 3) is that restore seems to drop the temporary variables created in 2), making step 5) impossible.

Maybe there's some tempfile workaround, but I can't think of the right way to do it. I feel like I'm missing something obvious.

edit: I'm using Stata 15 if it matters...
edit 2: I guess I can imagine some horribly complicated thing involving merging of a temporary ds, but the should be something easier, I'd think/hope.

[Nick Cox]

Someone may recognise this situation but I can't go beyond asking that you make this concrete, as I can't easily imagine how this could arise. However, in multidensity (SSC) recently I persuaded myself that the way to approach a problem was to create a set of extra variables that other commands could then use -- and which could easily be cleared out of the way too.

frames are also a framework (indeed) that may give you the flexibility you need but that can only be an incentive to upgrade to Stata 16.

[Danny Lempert]

Nick, Thanks for the reply. I will check out multidensity. In the meantime, let me try to make the example more concrete, though still boiled to its essentials.

My .ado file is a postestimation command, so there is a set of estimation results, let's say logit, in memory before the user runs my program.

As part of the syntax for my program, the user specifies (as relevant here), a variable on the rhs of the stored logit, let's call it x, and two values that x could take on, call them a and b.

The program calls predict twice, setting x, in turn, to a and b, creating two temporary variables to store those predicted probabilities, call them `tva' and `tvb'.

The program calculates a variable that is a function of `tva' and `tvb', call it z.

After the program ends, z should be added to the user's dataset, but the original values of x restored.


edit: OK, I think I figured it out. I should create a tempvar = x w/in the .ado file before calling predict, and replace x = tempvar once tva and tvb are created. There are actually other modifications of the data that I will need to reverse, without preserve, but those should be doable. Though I guess there is still some danger if the user presses break while the program is running....

[Danny Lempert]

btw, you are right, frames would make this really easy. damn it.

[Danny Lempert]

In case someone has a similar question later and finds this, here is what I ended up doing. As Nick suggested, it appears that frames would make this easier, but the following seems to work in a pinch if only Stata 15 (or older) is available. To simplify things, let's assume we want to "use" the temporary variables simply by giving them to the user in his dataset. (If you want to use them for other purposes after a restore, the below will require some modification, but the basic idea is the same.)

0) preserve data
0.5) before any modification of data, especially sorting or dropping any obss, create permanent variable n=_n using permname.
1) modify the user's data
2) for temporary variables that will need to be used after restore, create them as permanent variables instead, using permname.
2.5) keep n and all variables created using permname, sort on n, and save dataset using tempfile
3) restore the user's data
4) do some other stuff that requires the data as restored in 3)
4.5) merge in temporary dataset 1:1 on _n (=by observation)*, drop n.
5) end program, giving to the user the "temporary" variables created in 2), along with his dataset.


* this is a little scary, but using assert n=_n post-merge should ensure nothing went wrong.
edit: the nobreak command can be helpful in steps 4-5, too.

[daniel klein]

I hope I will not mess up things with some minor additions (that I have not tested).

In step 0.5, make sure to create n as long

generate long `n' = _n

In step 3, using

restore , preserve

might be safer and should not require nobreak; remember to

restore , not

before the program ends (successfully).


Last, implement an option that lets the user decide whether or not they actually want to keep the "temporary" variables.

[Danny Lempert]

Daniel, Thanks for the reply. Great reminder about long--that would have got me. Also, you are right about "restore, preserve" followed by "restore, not" at the appropriate points. Re your last point: yes, this whole thing only kicks in if the user specifies a generate(newvarname) option, but certainly that is a good reminder for others who may see this.