Dear Stata users,
I am working on a local server and running some grid search. My do-file is very basic:based on some parameter values, I generate simulated data and get the closest match to an observed data by some criteria. A bunch of gen, sort, replace, keep, define locals commands wrapped by "preserve" and "restore" for each of say 12 000 iteration.
Once I start my code, the following happens: the first few hundreds of iterations are lightning fast, take less than a second. Then, as time goes by, they get slower and slower. At the 1600th iteration it takes 15s or so per iteration!
If I start my code directly from the 1600th iteration, it processes it in the usual quick fashion for less than a second. So it is not that iterations themselves become more computationally intensive, it must be due to Stata's way of processing stuff.
One of my hopes was that maybe I am doing some other operations on the Company server that "steal" CPU, and this is the cause of my troubles. I tried using AWS. Set up a VM solely for the purpose of running this programme and nothing else. Same story, same pattern over time.
How could one escape from that? Is it possible at all? Is the programme storing some useless info that I can get rid of to improve performance?
Any opinion is appreciated.
I am working on a local server and running some grid search. My do-file is very basic:based on some parameter values, I generate simulated data and get the closest match to an observed data by some criteria. A bunch of gen, sort, replace, keep, define locals commands wrapped by "preserve" and "restore" for each of say 12 000 iteration.
Once I start my code, the following happens: the first few hundreds of iterations are lightning fast, take less than a second. Then, as time goes by, they get slower and slower. At the 1600th iteration it takes 15s or so per iteration!
If I start my code directly from the 1600th iteration, it processes it in the usual quick fashion for less than a second. So it is not that iterations themselves become more computationally intensive, it must be due to Stata's way of processing stuff.
One of my hopes was that maybe I am doing some other operations on the Company server that "steal" CPU, and this is the cause of my troubles. I tried using AWS. Set up a VM solely for the purpose of running this programme and nothing else. Same story, same pattern over time.
How could one escape from that? Is it possible at all? Is the programme storing some useless info that I can get rid of to improve performance?
Any opinion is appreciated.
Comment