outreg2 output really slow

[Zachary Feldman]

I'm having some strange issues with outreg2.

I'm using outreg2 inside a macro.

The code is:


reg `2' c.diff_from_normal b4.race_num i.gender_num i.sped_num i.grade i.meal i.title1_num i.gt_num i.migrant_num `3'#i.use_2005 i.english b6001001.lea, vce(cluster lea)
outreg2 using `4', replace keep(diff_from_normal) ctitle(A A A A)


I'm calling basically the same structure over different subsets of data about 80 times.
It had been and is still working perfectly fine on one computer, but on the other it is taking a really long time to output after each regression. The variable "b6001001.lea" is a very large set of fixed effects. If I cut these out outreg2 works fine.

It seems like it could be a memory issue, but I'm not sure what memory parameter to increase.

Thanks

[Zachary Feldman]

Still can't figure this out.
If I switch all the outreg2 commands to eststo/esttab commands, everything works fine. But with outreg2, STATA takes 10+ to write to the file. I can't stop the process either, I have to force quit STATA if I don't want to wait for it. I can switch everything to eststo/esttab, but I'd rather not have to switch all my code and the work flow for getting my tables formatted.

[Nick Cox]

It's hard to comment here, except to underline that

outreg2 (SSC), eststo (SJ/SSC), esttab (SJ/SSC) are user-written programs; hence you are expected to explain their provenance

outreg2 is a popular download, but its author is not an active member of Statalist, so whenever other users can't offer solutions you should think of contacting him directly

you've not given a reproducible example using data you supply or which we can all download.

Please study the FAQ, all the way to section 18, for advice on formulating questions.

I don't have any specific ideas on what is going on here; I don't use outreg2.

[Richard Williams]

I assume you have the latest versions of outreg2 and Stata on all machines involved. If you can fix the problem, great, but if not you should switch. I'd be surprised if you had written a ton of outreg2 commands before realizing the problem but maybe you did.

[Zachary Feldman]

Yep, latest outreg2 and STATA. The issue isn't that I didn't realize it was a problem, but that it doesn't work on a new computer (outreg2 continues to work fine on the old computer).
Looks like I'll just need to switch the rest of the code to eststo/esttab. Mostly hoped someone else had run into this problem and knew of a solution.

[Sebastien Bradley]

I was experiencing an identical problem with outreg2 in the last ~10 days, with outreg2 requiring extraordinarily long times to output a subset of estimates from regressions involving hundreds of dummy variable fixed effects. Likewise, this was only an issue on my machine running the latest version of Stata.
It seems, however, that the culprit is in the version of outreg2, for which an update was also evidently installed around 7/3/14. Reverting to the 9/30/11 version of outreg2.ado in the latest version of Stata restores the expected speed for outputting results.

[alejoforero]

Almost 10 years later, but if useful for anyone:

outreg2 is extremely slow with large data because it uses a lot of preserve-restore commands, and sorting with the esample variables (_est_...). But for exporting regressions most of these lines are useless and just slow down the execution.

A nice trick is to do this:

preserve
keep in 1/10
outreg2 [.......]
restore

This way it only uses one preserve-restore command, and the results are exactly the same since it is just exporting estimates results, not data from the dataset.

[Dhruv Jain DSE]

This actually works! Doesn't speed things up drastically but definitely the second best option!

Almost 10 years later, but if useful for anyone:

outreg2 is extremely slow with large data because it uses a lot of preserve-restore commands, and sorting with the esample variables (_est_...). But for exporting regressions most of these lines are useless and just slow down the execution.

A nice trick is to do this:

preserve
keep in 1/10
outreg2 [.......]
restore

This way it only uses one preserve-restore command, and the results are exactly the same since it is just exporting estimates results, not data from the dataset.

[alejoforero]

If you have a computer with a good ammount of memory and you use

set max preservemem 10g

with enough memory to save the current dataset in the preserve, then it should speed things up drastically.

[Ivan Gonzalez]

alejoforero Nice hack, but unfortunately it doesn't work whenever Descriptive Statistics are being calculated, i.e. for outreg2 with stats or eqkeep options since keep cuts the observations!
Any other trick in mind?

Thanks in advance!