Noisily Option and Computational Efficiency

[Heide Jackson]

Hello,

I am running some time intensive multi-level models on multiply imputed data using the mi estimate, mixed commands with Stata 18. My collaborator is interested in seeing the full set of models with the noisily option, and I was just curious, is noisily more computationally inefficient than the default set of commands or running quietly? If so, by how much does noisily increase the run time? Intuitively, this would seem likely, but I haven't found a full answer to this in the Stata help documentation. If anyone has an answer on this, it would be most appreciated.

Thanks!

[Clyde Schechter]

Well, I/O is slow compared to computation. I assume that you are logging your results, so you will engage two modes of I/O: console and log file on disk. That's got to add some time. The question is how much.

I tried an experiment running -regress price mpg trunk headroom- on the auto.dta 100 times, first quietly and then noisily, timing as we go. The 100 regressions done quietly took 1.04 seconds, and noisily they took 1.46 seconds. While you could say that this is a 40% increase in run time, I think it is more likely an additive than a multiplicative effect: it took an additional 0.42 seconds. Now, what you are doing is a more complicated calculation, and -mixed- can be orders of magnitude slower than -regress-. But I think the incremental time associated with -noisily- will still be small. -mixed- produces more output than -regress- does, so maybe an increment of a second or two for 100 replications. But I doubt it will be anything really noticeable.

To be honest, I don't know what your collaborator thinks he or she will learn from seeing these 100 regression outputs. But in light of the low cost, I wouldn't discourage curiosity here. The time he or she spends staring at all that output and trying to glean something useful from it will far exceed the incremental run time to generate the output.

[Heide Jackson]

Thanks Clyde. Really appreciate you running this case study. I was very curious as I haven't found anything on this in the Stata MP performance report and wanted to know how consequential or trivial the slow down would be.

[Clyde Schechter]

Well, I wouldn't expect to find an answer to this specific question in the MP performance report. While I suspect that multiple cores are used and there is parallel processing in the calculations of the -mixed- commands and within -mi estimate- itself, especially as -mi estimate- is "embarrassingly parallel." BUT, the display of output cannot be parallelized: there is only one Results window on the console and the output has to be ordered properly. So even if the output is created in parallel, there has to be some synchronization that, in effect, de-parallelizes the actual output process itself.

[Leonardo Guizzetti]

I agree with Clyde’s impressions about the output being additive to the overall runtime. I recently did some simulation work with mixed and my impressions are that any output to the results window added a negligible amount of time to putting the output to the results windows and log. This is because, for my situation, the mixed models took around a second to fit, which is orders of magnitude longer than the delay required to print the output. If I can run a smaller scale test for you, I’ll report back.

[Leonardo Guizzetti]

I just tested this (very informally) on a simulation with 1000 mixed models, suppressing output or displaying it to the results pane (no logging enabled). Timing was not distinguishable between the two display modes.