Issue with using -outreg2- in a loop

[Julia Brown]

Hello,

I am using the user-written command -outreg2- to output regression results to Excel. It is very possible that I should update my code with a different command for making regression tables, but that's a different post.

My regressions look something like

reg y treat1 treat2 treat3 [set of covariates]

I am running this 5 times for each of my outcome variables, but changing the covariates each time for robustness. After each regression, I am testing pairwise differences between treatment coefficients, and storing the results. I then output using outreg2. So the full thing looks like this:

local replace replace 

foreach y in $outcomes { 

forvalues i = 1/5 { 

reg y treat1 treat2 treat3 `covars`i'' 

test treat1=treat2 
local test12 = r(F) 
local test12p = r(p) 

test treat1=treat3 
local test13 = r(F) 
local test13p = r(p) 

​​​​​​​test treat2=treat3 
local test23 = r(F) 
local test23p = r(p) 

outreg2 using "regressions.xls", `replace' excel ctitle(`y') ///
addstat("Treat1 = Treat2",`test12', "pval",`test12p', ///
"Treat1 = Treat3",`test13', "pval",`test13p', ///
"Treat2 = Treat3",`test23', "pval",`test23p')

local replace 

}
}

This seemed like it was working fine, until I looked closely at my output and realized that the F stats and p-values for the pairwise tests in the addstat option were getting switched around in some of the results but not others. So for example, I was getting the p-value for the test of treat1=treat2 placed in the row for treat2=treat3, or vice versa. But just sometimes, not all the time, and never in the first regression for a given outcome variable, just in subsequent regressions with different covariates (ie when i = 1, not when i = 2-5).

So far, I have tried replacing the locals with scalars and also forcing Stata to evaluate the locals by adding "display "`test23'" right after the local is created (because it was suggested to me that maybe Stata wasn't evaluating the locals right away). Neither of these solved the problem.

(aside - ChatGPT told me this was a "known bug" with outreg2, but when I pushed it for references, it pointed me to a number of other posts on StataList that had nothing to do with this problem, so there's an object lesson in not trusting AI.)

Any thoughts on what might be happening would be very welcome! Thank you in advance.

[Julia Brown]

I solved my own problem - adding the solution here in case it's useful to anyone else.

Seems like outreg2 was getting confused by the fact that my pvalue rows all had the same name. Once I changed them to be unique, the weird behavior stopped. So my new outreg2 command is:

 outreg2 using "regressions.xls", `replace' excel ctitle(`y') ///
addstat("Treat1 = Treat2",`test12', "pval 1-2",`test12p', ///
"Treat1 = Treat3",`test13', "pval 1-3",`test13p', ///
"Treat2 = Treat3",`test23', "pval 2-3",`test23p')