[William Lisowski]

Hi Nikolay -

The good news is that I now understand what you uploaded and I've gotten your Grid.do to run on my system with minimal changes. The bad news is that my system is a MacBook Air, much slower than your systems, so the initial iterations take about 3 seconds. I will fire up the program when my system is otherwise idle (probably overnight) and we will see what happens for the later iterations.

I will report that I replaced the preserve/restore pair with an initial save to a temporary file before the loops begin, and a use, clear of the temporary file instead of restore, and it knocked a half-second off my loop time. I have no illusion that it will have solved the problem of the increasing loop time, however.

William

[Nikolay Antonov]

Hi William,

I also replaced preserve/restore with temp files that I get rid of afterwards.

I also rearranged the code in such a way that I now drop certain observations at each inner loop: for the condition of diff_theta I drop observations, save a temp file, go to the condition for CL, drop observations, save a temp file, go to the condition for DEN and so on.

Instead of doing all this within the innermost loop, I spread it now at the steps before that. As a result, I do an iteration for about .3s, which is pretty good in my world. Tested it on the fully-blown server and it behaves in a stable way. Even if it goes up by a factor of 3, that is still slightly less than a second, which, for a worst-case scenario, is not that bad at all.

At the end, I did not figure out a solution for the weird Stata behaviour and it would probably come back and bite me in the *** some other time, but for the time being I am fine.

I'd like to thank you for your time and suggestions. Take care

[William Lisowski]

Overnight my version of your Grid.do ran to completion. Each iteration took between 2.4 and 4.0 seconds, with no trend in time and no apparent correlation to any of the values of the four loops (b_CL, b_DEN, ic, id). So my results on the whole agree with yours.

Thanks for posing a thought-provoking problem, even though we weren't able to reach an absolute explanation for your original experience.

[William Lisowski]

I subsequently replaced the preserve/restore commands in my version of Grid.do and although it now ran more slowly, through 1200 iterations it showed no sign any incresing trend in time.