Append different T-tests using esttab

[Chibo Dike]

Please, I would like to append the results from several t-tests in one table using esttab which is from the Stata Journal. I have done the following:

sysuse auto

first, test that price=turn

quietly{
ttest price=turn
preserve
expand 2, g(new)
gen diff= cond(new, turn, price)
eststo m1: mean diff if !new
eststo m2: mean diff if new
eststo m3: estpost ttest diff, by(new)
esttab m1 m2 m3, se mlab(actual predicted ATE) coeflab(diff "private")
}

second, test that trunk=turn

quietly{
ttest trunk=turn
preserve
expand 2, g(new1)
gen diff1= cond(new1, turn, trunk)
eststo m4: mean diff1 if !new1
eststo m5: mean diff1 if new1
eststo m6: estpost ttest diff1, by(new1)
esttab m4 m5 m6, se mlab(actual predicted ATE) coeflab(diff1 "public")
}

To put the two tests in one table, I have done this:

esttab m1 m2 m3 m4 m5 m6, se mlab(actual predicted ATE) coeflab(diff "private" diff1 "public")

which yields:

---------------------------------------------------------------------------------------------------------
> ---
                      (1)             (2)             (3)             (4)             (5)             (6)
>    
                   actual       predicted             ATE              m4              m5              m6
>    
---------------------------------------------------------------------------------------------------------
> ---
private            6165.3***        39.65***       6125.6***                                            
>    
                  (342.9)         (0.511)         (342.9)                                                
>    

public                                                              13.76***        39.65***       -25.89
> ***
                                                                  (0.497)         (0.511)         (0.713)
>    
---------------------------------------------------------------------------------------------------------
> ---
N                      74              74             148              74              74             148
>    
---------------------------------------------------------------------------------------------------------
> ---
Standard errors in parentheses
* p<0.05, ** p<0.01, *** p<0.001

But this is not quite what I want. I want all test results placed vertically under the same 3 columns i.e actual, predicted and ATE (in other words m4, m5 and m6 respectively are 'actual', 'predicted' and 'ATE' from the second test). For simplicity this example invloves just two separate. But my work involves several of such tests.

[Andrew Musau]

Do everything in one go.

sysuse auto
preserve
expand 2, g(new)
gen diff1= cond(new, turn, price)
gen diff2= cond(new, turn, trunk)
eststo m1: mean diff1 diff2 if !new
eststo m2: mean diff1 diff2 if new
eststo m3: estpost ttest diff1 diff2, by(new)
esttab m1 m2 m3, se mlab(actual predicted ATE) coeflab(diff1 "private" diff2 "public")
restore

Res.:

. esttab m1 m2 m3, se mlab(actual predicted ATE) coeflab(diff1 "private" diff2 "public")

------------------------------------------------------------
                      (1)             (2)             (3)   
                   actual       predicted             ATE   
------------------------------------------------------------
private            6165.3***        39.65***       6125.6***
                  (342.9)         (0.511)         (342.9)   

public              13.76***        39.65***       -25.89***
                  (0.497)         (0.511)         (0.713)   
------------------------------------------------------------
N                      74              74             148   
------------------------------------------------------------
Standard errors in parentheses
* p<0.05, ** p<0.01, *** p<0.001

[Chibo Dike]

Thanks Andrew. I failed to state in my initial post that the tests are based on different observations or groups within the same data set. For example ttest price=turn is based on n=74 while ttest speed=miles based on n=52. Here's what i'm talking:

sysuse auto, clear
gen speed=turn
replace speed=. if foreign==1
gen miles=weight
replace miles=. if foreign==1

Now, say we want to append the result from two tests: test price=turn and speed=miles

first, as I said above note that

 ttest turn=price

yields

Paired t test
------------------------------------------------------------------------------
Variable |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
    turn |      74    39.64865    .5114145    4.399354     38.6294     40.6679
   price |      74    6165.257    342.8719    2949.496    5481.914      6848.6
---------+--------------------------------------------------------------------
    diff |      74   -6125.608    342.7139    2948.137   -6808.636    -5442.58
------------------------------------------------------------------------------
     mean(diff) = mean(turn - price)                              t = -17.8738
 Ho: mean(diff) = 0                              degrees of freedom =       73

and that

ttest speed=miles

yields

Paired t test
------------------------------------------------------------------------------
Variable |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
   speed |      52    41.44231    .5502046    3.967582    40.33773    42.54689
   miles |      52    3317.115     96.4296    695.3637    3123.525    3510.706
---------+--------------------------------------------------------------------
    diff |      52   -3275.673    95.99306    692.2158   -3468.387   -3082.959
------------------------------------------------------------------------------
     mean(diff) = mean(speed - miles)                             t = -34.1241
 Ho: mean(diff) = 0                              degrees of freedom =       51

Now, adapting your code to run the two tests in one go, I did this:

preserve
expand 2, g(new)
gen diff1= cond(new, turn, price)
gen diff2= cond(new, miles, speed)
eststo m1: mean diff1 diff2 if !new
eststo m2: mean diff1 diff2 if new
eststo m3: estpost ttest diff1 diff2, by(new)
esttab m1 m2 m3, se mlab(actual predicted ATE) coeflab(diff1 "private" diff2 "public")
restore

which yields:

------------------------------------------------------------
                      (1)             (2)             (3)   
                   actual       predicted             ATE   
------------------------------------------------------------
private            6072.4***        41.44***       6125.6***
                  (429.5)         (0.550)         (342.9)   

public              41.44***       3317.1***      -3275.7***
                  (0.550)         (96.43)         (96.43)   
------------------------------------------------------------
N                      52              52             148   
------------------------------------------------------------
Standard errors in parentheses
* p<0.05, ** p<0.01, *** p<0.001

However, the figures in the private row are wrong. They are calculated based on N=52 of the second test. What i'm trying to do is to stack the results of several independent tests (and then possibly run esttab along with "noobs" since N row is not very meaningful in this context).
.

[Andrew Musau]

Look at appendmodels by Benn Jann. Post back if you are unable to figure it out.

[Chibo Dike]

Thanks Andrew. If I'm honest, that page is indecipherable to me. Perhaps it's not made for t-tests. Maybe I'll just run the different tests and then manually combine them in one table when I'm compiling the final version of my paper to send out.

[Andrew Musau]

I see the problem. The ttest results do not include a variance matrix, but a standard errors matrix. The following should fix it and uses erepost from SSC.

capt prog drop appendmodels
*! version 1.0.0  14aug2007  Ben Jann
program appendmodels, eclass
    // using first equation of model
    version 8
    syntax namelist
    tempname b V tmp
    foreach name of local namelist {
        qui est restore `name'
        mat `tmp' = e(b)
        local eq1: coleq `tmp'
        gettoken eq1 : eq1
        mat `tmp' = `tmp'[1,"`eq1':"]
        local cons = colnumb(`tmp',"_cons")
        if `cons'<. & `cons'>1 {
            mat `tmp' = `tmp'[1,1..`cons'-1]
        }
        mat `b' = nullmat(`b') , `tmp'
        mat `tmp' = e(V)
        mat `tmp' = `tmp'["`eq1':","`eq1':"]
        if `cons'<. & `cons'>1 {
            mat `tmp' = `tmp'[1..`cons'-1,1..`cons'-1]
        }
        capt confirm matrix `V'
        if _rc {
            mat `V' = `tmp'
        }
        else {
            mat `V' = ///
            ( `V' , J(rowsof(`V'),colsof(`tmp'),0) ) \ ///
            ( J(rowsof(`tmp'),colsof(`V'),0) , `tmp' )
        }
    }
    local names: colfullnames `b'
    mat coln `V' = `names'
    mat rown `V' = `names'
    eret post `b' `V'
    eret local cmd "whatever"
end

cap prog drop repbV
program repbV, eclass
erepost b= b, rename
erepost V= V, rename
end
ssc install erepost, replace

sysuse auto, clear
gen speed=turn
replace speed=. if foreign==1
gen miles=weight
replace miles=. if foreign==1
preserve
expand 2, g(new)
gen diff1= cond(new, turn, price)
eststo m1: mean diff1 if !new
eststo m2: mean diff1 if new
estpost ttest diff1, by(new)
mat b= e(b)
mat V= (e(se)[1,1])^2
qui mean diff1
repbV
est sto m3 
gen diff2= cond(new, miles, speed)
eststo m4: mean diff2 if !new
eststo m5: mean diff2 if new
estpost ttest diff2, by(new)
mat b= e(b)
mat V= (e(se)[1,1])^2
qui mean diff2
repbV
est sto m6

eststo M1: appendmodels m1 m4
eststo M2: appendmodels m2 m5
eststo M3: appendmodels m3 m6
esttab M1 M2 M3, se mlab(actual predicted ATE) coeflab(diff1 "private" diff2 "public") noobs

Res.:

. esttab M1 M2 M3, se mlab(actual predicted ATE) coeflab(diff1 "private" diff2 "
> public") noobs

------------------------------------------------------------
                      (1)             (2)             (3)   
                   actual       predicted             ATE   
------------------------------------------------------------
private            6165.3***        39.65***       6125.6***
                  (342.9)         (0.511)         (342.9)   

public              41.44***       3317.1***      -3275.7***
                  (0.550)         (96.43)         (96.43)   
------------------------------------------------------------
Standard errors in parentheses
* p<0.05, ** p<0.01, *** p<0.001

[Chibo Dike]

Thanks a lot. Unfortunately I'm unable to reproduce this on my computer (I use stata 13). Running each code line by line, I see that the problem is with the line "mat V= (e(se)[1,1])^2".

mat V= (e(se)[1,1])^2
invalid syntax
r(198);

Same problem when I adapt the code to my own data. I've restarted stata and my computer but the error persists. Perhaps it is related to this point in the erepost help:

Technical note: After applying erepost the original command (or estimates replay) may not be able to replay the output.

But i don't know how to proceed from here.

[Andrew Musau]

If you do not state your version of Stata, the assumption is that you have the latest version (FAQ Advice #12).

mat V= (e(se)[1,1])^2

You cannot refer to an -e()- matrix as part of an expression in versions 15 or earlier. So you need the following sequence:

mat se= e(se)
mat V= (se[1,1])^2

[Chibo Dike]

Thanks. Sorry this is dragging on. I've plugged in the two mats you gave. There's another error. Everything works until the repbV line:

. repbV
invalid syntax
r(198);

end of do-file

r(198);

Perhaps changes need to be made here too? (I've already tried changing a thing or two there, but I really don't know).

cap prog drop repbV
program repbV, eclass
erepost b= b, rename
erepost V= V, rename
end

[Andrew Musau]

Copy exactly the following, paste directly into a do-file and run. Then copy and paste all the output from the command window. I do not have version 13 to test, but this will rule out some invisible characters created from copying and pasting the code into an external editor and then copying again into the do-file.

capt prog drop appendmodels
*! version 1.0.0  14aug2007  Ben Jann
program appendmodels, eclass
    // using first equation of model
    version 8
    syntax namelist
    tempname b V tmp
    foreach name of local namelist {
        qui est restore `name'
        mat `tmp' = e(b)
        local eq1: coleq `tmp'
        gettoken eq1 : eq1
        mat `tmp' = `tmp'[1,"`eq1':"]
        local cons = colnumb(`tmp',"_cons")
        if `cons'<. & `cons'>1 {
            mat `tmp' = `tmp'[1,1..`cons'-1]
        }
        mat `b' = nullmat(`b') , `tmp'
        mat `tmp' = e(V)
        mat `tmp' = `tmp'["`eq1':","`eq1':"]
        if `cons'<. & `cons'>1 {
            mat `tmp' = `tmp'[1..`cons'-1,1..`cons'-1]
        }
        capt confirm matrix `V'
        if _rc {
            mat `V' = `tmp'
        }
        else {
            mat `V' = ///
            ( `V' , J(rowsof(`V'),colsof(`tmp'),0) ) \ ///
            ( J(rowsof(`tmp'),colsof(`V'),0) , `tmp' )
        }
    }
    local names: colfullnames `b'
    mat coln `V' = `names'
    mat rown `V' = `names'
    eret post `b' `V'
    eret local cmd "whatever"
end

cap prog drop repbV
program repbV, eclass
erepost b= b, rename
erepost V= V, rename
end
ssc install erepost, replace

sysuse auto, clear
gen speed=turn
replace speed=. if foreign==1
gen miles=weight
replace miles=. if foreign==1
preserve
expand 2, g(new)
gen diff1= cond(new, turn, price)
eststo m1: mean diff1 if !new
eststo m2: mean diff1 if new
estpost ttest diff1, by(new)
mat b= e(b)
mat se= e(se)
mat V= (se[1,1])^2
qui mean diff1
repbV
est sto m3
gen diff2= cond(new, miles, speed)
eststo m4: mean diff2 if !new
eststo m5: mean diff2 if new
estpost ttest diff2, by(new)
mat b= e(b)
mat se= e(se)
mat V= (se[1,1])^2
qui mean diff2
repbV
est sto m6

eststo M1: appendmodels m1 m4
eststo M2: appendmodels m2 m5
eststo M3: appendmodels m3 m6
esttab M1 M2 M3, se mlab(actual predicted ATE) coeflab(diff1 "private" diff2 "public") noobs

[Chibo Dike]

It works! I adapted it to my own data and it works as desired. As you have asked to see the result on my computer, there you go:

do "C:\Users\Owner\AppData\Local\Temp\STD00000000.tmp"

. capt prog drop appendmodels

. *! version 1.0.0  14aug2007  Ben Jann
. program appendmodels, eclass
  1.     // using first equation of model
.     version 8
  2.     syntax namelist
  3.     tempname b V tmp
  4.     foreach name of local namelist {
  5.         qui est restore `name'
  6.         mat `tmp' = e(b)
  7.         local eq1: coleq `tmp'
  8.         gettoken eq1 : eq1
  9.         mat `tmp' = `tmp'[1,"`eq1':"]
 10.         local cons = colnumb(`tmp',"_cons")
 11.         if `cons'<. & `cons'>1 {
 12.             mat `tmp' = `tmp'[1,1..`cons'-1]
 13.         }
 14.         mat `b' = nullmat(`b') , `tmp'
 15.         mat `tmp' = e(V)
 16.         mat `tmp' = `tmp'["`eq1':","`eq1':"]
 17.         if `cons'<. & `cons'>1 {
 18.             mat `tmp' = `tmp'[1..`cons'-1,1..`cons'-1]
 19.         }
 20.         capt confirm matrix `V'
 21.         if _rc {
 22.             mat `V' = `tmp'
 23.         }
 24.         else {
 25.             mat `V' = ///
>             ( `V' , J(rowsof(`V'),colsof(`tmp'),0) ) \ ///
>             ( J(rowsof(`tmp'),colsof(`V'),0) , `tmp' )
 26.         }
 27.     }
 28.     local names: colfullnames `b'
 29.     mat coln `V' = `names'
 30.     mat rown `V' = `names'
 31.     eret post `b' `V'
 32.     eret local cmd "whatever"
 33. end

. 
. cap prog drop repbV

. program repbV, eclass
  1. erepost b= b, rename
  2. erepost V= V, rename
  3. end

. ssc install erepost, replace
checking erepost consistency and verifying not already installed...
all files already exist and are up to date.

. 
. sysuse auto, clear
(1978 Automobile Data)

. gen speed=turn

. replace speed=. if foreign==1
(22 real changes made, 22 to missing)

. gen miles=weight

. replace miles=. if foreign==1
(22 real changes made, 22 to missing)

. preserve

. expand 2, g(new)
(74 observations created)

. gen diff1= cond(new, turn, price)

. eststo m1: mean diff1 if !new

Mean estimation                     Number of obs    =      74

--------------------------------------------------------------
             |       Mean   Std. Err.     [95% Conf. Interval]
-------------+------------------------------------------------
       diff1 |   6165.257   342.8719      5481.914      6848.6
--------------------------------------------------------------

. eststo m2: mean diff1 if new

Mean estimation                     Number of obs    =      74

--------------------------------------------------------------
             |       Mean   Std. Err.     [95% Conf. Interval]
-------------+------------------------------------------------
       diff1 |   39.64865   .5114145       38.6294     40.6679
--------------------------------------------------------------

. estpost ttest diff1, by(new)

             |      e(b)   e(count)      e(se)       e(t)    e(df_t)     e(p_l)       e(p)     e(p_u)     e(N_1)    e(mu_1) 
-------------+--------------------------------------------------------------------------------------------------------------
       diff1 |  6125.608        148   342.8723   17.86557        146          1   1.45e-38   7.24e-39         74   6165.257 

             |    e(N_2)    e(mu_2) 
-------------+----------------------
       diff1 |        74   39.64865 

. mat b= e(b)

. mat se= e(se)

. mat V= (se[1,1])^2

. qui mean diff1

. repbV

. est sto m3

. gen diff2= cond(new, miles, speed)
(44 missing values generated)

. eststo m4: mean diff2 if !new

Mean estimation                     Number of obs    =      52

--------------------------------------------------------------
             |       Mean   Std. Err.     [95% Conf. Interval]
-------------+------------------------------------------------
       diff2 |   41.44231   .5502046      40.33773    42.54689
--------------------------------------------------------------

. eststo m5: mean diff2 if new

Mean estimation                     Number of obs    =      52

--------------------------------------------------------------
             |       Mean   Std. Err.     [95% Conf. Interval]
-------------+------------------------------------------------
       diff2 |   3317.115    96.4296      3123.525    3510.706
--------------------------------------------------------------

. estpost ttest diff2, by(new)

             |      e(b)   e(count)      e(se)       e(t)    e(df_t)     e(p_l)       e(p)     e(p_u)     e(N_1)    e(mu_1) 
-------------+--------------------------------------------------------------------------------------------------------------
       diff2 | -3275.673        104   96.43117  -33.96903        102   1.01e-57   2.03e-57          1         52   41.44231 

             |    e(N_2)    e(mu_2) 
-------------+----------------------
       diff2 |        52   3317.115 

. mat b= e(b)

. mat se= e(se)

. mat V= (se[1,1])^2

. qui mean diff2

. repbV

. est sto m6

. 
. eststo M1: appendmodels m1 m4

. eststo M2: appendmodels m2 m5

. eststo M3: appendmodels m3 m6

. esttab M1 M2 M3, se mlab(actual predicted ATE) coeflab(diff1 "private" diff2 "public") noobs

------------------------------------------------------------
                      (1)             (2)             (3)   
                   actual       predicted             ATE   
------------------------------------------------------------
private            6165.3***        39.65***       6125.6***
                  (342.9)         (0.511)         (342.9)   

public              41.44***       3317.1***      -3275.7***
                  (0.550)         (96.43)         (96.43)   
------------------------------------------------------------
Standard errors in parentheses
* p<0.05, ** p<0.01, *** p<0.001

. 
end of do-file