Simulate - any way to record random values used in each iteration?

[Trevor Zink]

Stata's -simulate- command draws values from a distribution built around point estimates. When used with -forecast solve-, Stata creates distributions around estimated coefficients in two or more structural equations based on the standard error of each coefficient estimate. Then it draws from those distributions each iteration and solves the model using those values. Quite a nice tool, really.

However, I need to know specific information about the random coefficients -simulate- uses in each iteration. For instance, for a given iteration, I need to know the values of all the coefficients in the model to see if make theoretical sense (and eliminate them if not). According to the manual, I can't see any way to extract the actual random values used in each iteration; only the model solutions each iteration are saved.

Does anyone know a way to solve the actual values input into the model each iteration? It seems like the -simulate- command and the -forecast solve- with simulate option are essentially the same, so if you know how to do this with the simulate command, I'd be happy to hear that too.

[Nick Cox]

I guess you are looking in the wrong direction. simulate is a command to run some other command, and that other command is typically user-written. If there is a need to check a distribution, or modify it in some way, then you can delegate that to the other command. That might well mean just returning missing if a sample draw didn't meet your criteria. I guess that if you wanted to keep track of the distributions you could post them somewhere. That's rarely done for many excellent reasons, but off-hand I know of nothing which rules that out.

Alternatively, just write your own loop that loops over lots of simulations. In qenv (SSC), for example, we used Mata functions directly, and made no use of simulate.

[Maarten Buis]

If Trevor wants to look at the code we used in qenv, then that is available as the ancillary files qenv_mata9.do and qenv_mata10.do. He can get those by first typing ssc describe qenv and than net get qenv .

[Trevor Zink]

I don't think I'm entirely looking in the wrong direction, as -simulate- is also an option to -forecast solve-, and as far as I can tell, it calls the same basic functions as the regular -simulate- command. If there were some way to force Stata to capture the random values for each draw in the base -simulate-, perhaps a similar method could be used with -forecast solve-
(I've learned that asking questions directly about -forecast- on here tends not to yield any responses).

If it's true that there's no way to capture the actual random values used in each iteration, that seems slightly odd. Stata certainly has to know these values at some point, as it first generates them from the specified distribution, and then plugs them in to some other command or expression--why not also record them at that time, or at least provide that to the user as an option?

[Nick Cox]

From a quick look at the code the link is just one of using the same words.

forecast solve certainly has a simulate() option but it does not appear to call the simulate command at all. So, it appears that you are restricted to whatever flexibility is provided directly by forecast solve, or obliged to hack out some quite different route.

Alternatively, do you have other evidence for your opening statement, which I confess I took on trust and did not investigate, that

Stata's -simulate- command draws values from a distribution built around point estimates. When used with -forecast solve-, Stata creates distributions around estimated coefficients in two or more structural equations based on the standard error of each coefficient estimate. Then it draws from those distributions each iteration and solves the model using those values.

?

As you have remarked elsewhere, there does not seem to be much experience with the forecast command among people active on Statalist. I have never used it. I surmise darkly that the Tao is right here: Those who know do not speak; those who speak do not know.

[Clyde Schechter]

Stata certainly has to know these values at some point, as it first generates them from the specified distribution, and then plugs them in to some other command or expression

Actually, that's not quite true. The -simulate- command itself does not generate any random values of anything. The program or command called by simulate is responsible for doing that. All that -simulate- does is call it repeatedly and save (some of) the values returned by that program or command into a tempfile. So, if you want to capture the random coefficients that are being generated, you will need to hack -forecast- so that it returns those coefficients somewhere in r() or e() each time it runs. Then you can add the relevant returned values to the arguments of the -simulate- command that calls it, and you will have them.

As I know nothing about -forecast- itself, I can't, however, offer any advice as to how to do that.

I guess I'm not really saying anything that Nick didn't already say in #2, but maybe this level of detail is more helpful to you?

[Trevor Zink]

Ah. My Stata programming knowledge isn't sufficient to tell me that -forecast solve- with the simulate option doesn't actually call the -simulate- command. They must have implemented it separately (for reasons unknown). Certainly my description of how the simulate option works, which you quoted, is true. That much is stated in the PDF manual. But no option is given in the forecast command to capture the random values used.
Looks like I'll have to build the distributions and do the monte carlo more or less by hand.
Thanks for the insight.