Storing and exporting results from two-sample t tests

[Keanan Gleason]

Hi,

I'm trying to export a table of two-sided p-values generated from two-sample t tests for about two dozen variables and three groups. Let's say I have variables a, b, c, and d, and groups 1, 2, and 3. For each variable, I'd like to conduct two-sample t tests for all pairwise combinations of the groups. Here is code to do this:

foreach group in 1 2 3 {
    foreach var in a b c {
        ttest `var', by(`group')
    }
}

I'd then like to save the two-sided p-value from each test and put it into a table where each column is a different pairwise combination (i.e. 1-2, 1-3, and 2-3) and each row is a different variable. The values in the table would be the p-values. Using esttab, I'd like to export this table into .tex format. I don't have code to show for either saving the p-values or exporting the table as I've tried a few different things using eststo, estpost, and estadd, but thus far none have worked.

I know that the command ttest stores the estimation results and I will need to use r(p), but I'm unsure of how to put it into either a local or a new variable that takes on a different value depending on the pairwise combination.

I'm using Stata 15.1. Any help would be greatly appreciated. I haven't posted on statalist many times before so apologies if there are any posting rules I missed.

Thanks,

Keanan

[Clyde Schechter]

The following code creates a toy demonstration data set of the kind you describe and does the calculations you ask for, almost:

//  CREATE A DEMONSTRATION DATA SET
clear*
set obs 100
set seed 1234

foreach x in a b c {
    gen `x' = runiform()
}
gen group = mod(_n, 3) + 1

//  CREATE A DATA SET THAT CONTAINS THE TABLE OF T-TEST P-VALUES
capture postutil clear
tempfile ttest_results
postfile handle str32 varname float(p12 p13 p23) using `ttest_results'
assert inlist(group, 1, 2, 3)
foreach x of varlist a b c {
    ttest `x' if group != 3, by(group)
    local p12 = r(p)
    ttest `x' if group != 2, by(group)
    local p13 = r(p)
    ttest `x' if group != 1, by(group)
    local p23 = r(p)
    post handle ("`x'") (`p12') (`p13') (`p23')
}
postclose handle

use `ttest_results', clear
list, noobs clean

As I am not an -esttab- user, I haven't used it here to manage these results. You can, however use -export delimited- at the end of this code to save the results in a text file, formatted to your taste.

In the future, when asking for help with code, please provide example data, and use the -dataex- command to do that. If you are running version 16 or a fully updated version 15.1 or 14.2, -dataex- is already part of your official Stata installation. If not, run -ssc install dataex- to get it. Either way, run -help dataex- to read the simple instructions for using it. -dataex- will save you time; it is easier and quicker than typing out tables. It includes complete information about aspects of the data that are often critical to answering your question but cannot be seen from tabular displays or screenshots. It also makes it possible for those who want to help you to create a faithful representation of your example to try out their code, which in turn makes it more likely that their answer will actually work in your data.

[Rich Goldstein]

as far as getting them into locals, add the following directly after your ttest statement:

local p`var'`group' = r(p)

you can then check them and look at them via (just showing one example)

mac li _pa1

[Keanan Gleason]

Thank you both for your suggestions. Clyde, I adapted your code and it worked perfectly. I will figure out a way to export it to a .tex file. I really appreciate the fast responses and will be sure to include example data in future posts.

Keanan

[Nader Mehri]

The following code creates a toy demonstration data set of the kind you describe and does the calculations you ask for, almost:

// CREATE A DEMONSTRATION DATA SET
clear*
set obs 100
set seed 1234

foreach x in a b c {
gen `x' = runiform()
}
gen group = mod(_n, 3) + 1

// CREATE A DATA SET THAT CONTAINS THE TABLE OF T-TEST P-VALUES
capture postutil clear
tempfile ttest_results
postfile handle str32 varname float(p12 p13 p23) using `ttest_results'
assert inlist(group, 1, 2, 3)
foreach x of varlist a b c {
ttest `x' if group != 3, by(group)
local p12 = r(p)
ttest `x' if group != 2, by(group)
local p13 = r(p)
ttest `x' if group != 1, by(group)
local p23 = r(p)
post handle ("`x'") (`p12') (`p13') (`p23')
}
postclose handle

use `ttest_results', clear
list, noobs clean

As I am not an -esttab- user, I haven't used it here to manage these results. You can, however use -export delimited- at the end of this code to save the results in a text file, formatted to your taste.

In the future, when asking for help with code, please provide example data, and use the -dataex- command to do that. If you are running version 16 or a fully updated version 15.1 or 14.2, -dataex- is already part of your official Stata installation. If not, run -ssc install dataex- to get it. Either way, run -help dataex- to read the simple instructions for using it. -dataex- will save you time; it is easier and quicker than typing out tables. It includes complete information about aspects of the data that are often critical to answering your question but cannot be seen from tabular displays or screenshots. It also makes it possible for those who want to help you to create a faithful representation of your example to try out their code, which in turn makes it more likely that their answer will actually work in your data.

Hi Clyde,
Using your suggested code, I developed the code below to run a set of ttest followed by effect size estimations. I would like to store and save the outputs (variable names, sample size for each group, p-values and correspodning ESs) in a .dta file format but I got an error of "invalid syntax post: above message corresponds to expression 2, variable p".
Thanks for your advice!
Nader

capture postutil clear
tempfile ttest_results
postfile handle str32 varname float(p`x') using `ttest_results'

foreach x of varlist a b c d e f g {
ttest `x' , by(AttendedFRP_new)
local p`x'= r(p)
esize twosample `x' , by(AttendedFRP_new) cohensd hedgesg

post handle ("`x'") (`px')
}
postclose handle
use `ttest_results', clear
list, noobs clean

[Clyde Schechter]

Your -postfile- command contains -float(p`x')-, but `x' is undefined at that point, so this just becomes -float(p)-. This does not actual harm in this instance., but it would be more direct to just write it as -float(p)- in the first place.

The trouble you are getting, however, arises from your -post handle- command. There is no local macro `px' defined. You did define a local macro p`x' in a preceding command. To refer to it, you must write `p`x''.

So it all becomes:

capture postutil clear
tempfile ttest_results
postfile handle str32 varname float(p) using `ttest_results'

foreach x of varlist a b c d e f g {
    ttest `x' , by(AttendedFRP_new)
    local p`x'= r(p)
    esize twosample `x' , by(AttendedFRP_new) cohensd hedgesg

    post handle ("`x'") (`p`x'')
}
postclose handle
use `ttest_results', clear
list, noobs clean

To get more output including the other results of -ttest- and the effect size from -esize- is a little more complicated because you are combining results from two different commands, so you must build up your -post handle- command in stages.

capture postutil clear
tempfile ttest_results
postfile handle str32 varname long(N1 N2) float(p cohend hedgesg) using `ttest_results'

foreach x of varlist a b c d e f g {
    ttest `x' , by(AttendedFRP_new)
    local topost ("`x'") (`r(N_1)') (`r(N_2)') (`r(p)')
    esize twosample `x' , by(AttendedFRP_new) cohensd hedgesg
    local topost `topost' (`r(d)') (`r(g)')
    post handle `topost'
}
postclose handle
use `ttest_results', clear
list, noobs clean

In the future, when asking for help with code, please use the -dataex- command and show example data. Although sometimes, as here, it is possible to give an answer that has a reasonable probability of being correct, this is usually not the case. Moreover, such answers are necessarily based on experience-based guesses or intuitions about the nature of your data. When those guesses are wrong, both you and the person trying to help you have wasted their time as you end up with useless code. To avoid this, a -dataex- based example provides all of the information needed to develop and test a solution.

[Nader Mehri]

Thanks! This is helpful and works perfectly. I was trying to include the t value in the output by adding
(`r(t)') in the code but got an error.

[Clyde Schechter]

It's not very helpful to just say you got an error. What error in particular? And from which command did it arise?

My best guess is that you may have just add (`r(t)') to the -post handle- command or to one of the -local topost- commands. But you also have to create a place for it in the -postfile- command. And the (`r(t)') would have to be added to the first -local topost- command, because once you run -esize-, -r(t)- is lost, and (`r(t)') will just be (), which is not allowed in -frame post-.

Try this:

postfile handle str32 varname long(N1 N2) float(t p cohend hedgesg) using `ttest_results'

foreach x of varlist a b c d e f g {
    ttest `x' , by(AttendedFRP_new)
    local topost ("`x'") (`r(N_1)') (`r(N_2)') (`r(t)') (`r(p)')
    esize twosample `x' , by(AttendedFRP_new) cohensd hedgesg
    local topost `topost' (`r(d)') (`r(g)')
    post handle `topost'
}