The effect of high-clock/high-IPC cores on computation time of Conley standard errors in an IV regression (acreg)

[Peter Heineke]

Hello everyone,

I have a conceptual question about Stata. I have a dataset with 7 million observations that represents grid cells over a part of a country. If I now use the acreg command to estimate my standard errors, it takes about 20 hours to run on a MacBook Pro M1 with 10 CPU cores. I am wondering if I can speed this up by renting a remote server with a machine that has higher-clock/higher-IPC cores to speed things up. Since I only have StataMP 10, I can only leverage 10 cores. Thus, I am wondering if higher quality cores would bring time improvements. But I lack sufficient knowledge of how Stata (and also the acreg command) works under the hood to understand if that would help. Could someone offer advice on this? Many thanks in advance!

[Daniel Feenberg]

There are clues on the Mac that you can use to estimate the liklihood that more or faster cores would help. Run the OS command -top- in a terminal window while -acreg- is running. How much CPU does -acreg- use? If cpu percentage is 900, then you know more cores will speed it up. If the cpu percentage hovers around 20, or any value less than 100 then you know the procedure is I/O bound and more cores won't help. You probably want to test with a small subset of the data, so you can observe the full lifetime of the procedure. Higher clock rates would help a CPU bound job, but generally more money buys you more cores, not faster cores.

https://www.stata.com/statamp/perfor...ort/report.pdf has information on this topic for many procedures, but not -acreg-.