Help with collect stars

[ghoetker]

I am struggling to get stars to show up when reporting the results of a ttest and hope someone can help/explain.

Consider the following, which is based on the examples of the TABLES manual.

 sysuse auto, clear
(1978 automobile data)

. collect: ttest mpg, by(foreign )

Two-sample t test with equal variances
------------------------------------------------------------------------------
   Group |     Obs        Mean    Std. err.   Std. dev.   [95% conf. interval]
---------+--------------------------------------------------------------------
Domestic |      52    19.82692     .657777    4.743297    18.50638    21.14747
 Foreign |      22    24.77273     1.40951    6.611187    21.84149    27.70396
---------+--------------------------------------------------------------------
Combined |      74     21.2973    .6725511    5.785503     19.9569    22.63769
---------+--------------------------------------------------------------------
    diff |           -4.945804    1.362162               -7.661225   -2.230384
------------------------------------------------------------------------------
    diff = mean(Domestic) - mean(Foreign)                         t =  -3.6308
H0: diff = 0                                     Degrees of freedom =       72

    Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
 Pr(T < t) = 0.0003         Pr(|T| > |t|) = 0.0005          Pr(T > t) = 0.9997

. collect stars _r_p 0.01 "***" 0.05 "** " 0.1 "* " 1 " ", attach(_r_mu_diff)


. collect layout (cmdset) (result[mu_diff stars])

Collection: default
      Rows: cmdset
   Columns: result[mu_diff stars]
   Table 1: 1 x 1

-----------------------
  | Difference of means
--+--------------------
1 |           -4.945804
-----------------------

Given that the difference is definitely significant, I'd expect to see stars in the table. Yet, I don't.

Many thanks for any help.

Glenn

[Daniel Schaefer]

Hi Glenn,

Edit: Possible answer in #3.

Unfortunately, I can't seem to reproduce your results in Stata 18. The issue comes a bit out of left field: I don’t see that the ttest command returns the difference of means.

sysuse auto, clear
collect clear
collect: ttest mpg, by(foreign )
return list

. return list

scalars:
              r(level) =  95
                 r(sd) =  5.785503209735139
               r(sd_2) =  6.611186898567625
               r(sd_1) =  4.743297247514701
                 r(se) =  1.362162113622176
                r(p_u) =  .9997372920330829
                r(p_l) =  .0002627079669171
                  r(p) =  .0005254159338342
                  r(t) =  -3.630848447731832
               r(df_t) =  72
               r(mu_2) =  24.77272727272727
                r(N_2) =  22
               r(mu_1) =  19.82692307692308
                r(N_1) =  52

.
end of do-file

Therefore mu_diff doesn't show up in the tags.

. collect levelsof result

Collection: default
 Dimension: result
    Levels: N_1 N_2 df_t level mu_1 mu_2 p p_l p_u sd sd_1 sd_2 se t

Although it will appear as a tag if I try to use mu_diff in a layout, it is empty and will not produce a valid result.

. collect layout (cmdset) (result[mu_diff])
(level mu_diff of dimension result not found)

Collection: default
      Rows: cmdset
   Columns: result[mu_diff]

Your layout specification does not identify any items.

. collect levelsof result

Collection: default
 Dimension: result
    Levels: N_1 N_2 df_t level mu_1 mu_2 mu_diff p p_l p_u sd sd_1 sd_2 se stars t

If I try to run your code start to finish on my end, I get the same Your layout specification does not identify any items message as above. I don't seem to have any trouble attaching stars to values other than mu_diff on my end. Here I'll just attach the stars to the first mean.

sysuse auto, clear
collect clear
collect: ttest mpg, by(foreign )
collect stars p 0.01 "***" 0.05 "** " 0.1 "* " 1 " ", attach(mu_1)
collect layout (cmdset) (result[mu_1])
collect levelsof result

. collect layout (cmdset) (result[mu_1])

Collection: default
      Rows: cmdset
   Columns: result[mu_1]
   Table 1: 1 x 1

----------------------------
  | x₁ mean for population 1
--+-------------------------
1 |              19.82692***
----------------------------

Obviously not what you are looking for, but the code above seems to demonstrate the principle. I'm not sure why you are able to access the mu_diff return value when I am not, but perhaps we are on different Stata versions. On my end, I can work around the missing return value and get the expected stars like so.

sysuse auto, clear
collect clear
collect: ttest mpg, by(foreign)
local diff = r(mu_1) - r(mu_2)
collect get mu_diff = `diff', tags(cmdset[1])
collect stars p 0.01 "***" 0.05 "** " 0.1 "* " 1 " ", attach(mu_diff)
collect layout (cmdset) (result[mu_diff])

. collect layout (cmdset) (result[mu_diff])

Collection: default
      Rows: cmdset
   Columns: result[mu_diff]
   Table 1: 1 x 1

----------------
  |      mu_diff
--+-------------
1 | -4.945804***
----------------

[Daniel Schaefer]

I use the tag names rather than the _r_ syntax in collect stars. When I use your syntax I get the same result as you. I lose the stars too. I'd guess that's the issue and you need this:

collect stars p 0.01 "***" 0.05 "** " 0.1 "* " 1 " ", attach(mu_diff)

But it is hard to say for sure because I can't reproduce your code precisely. I have seen the syntax you use in the docs, and I'm honestly not clear as to why it doesn't work here, but I think the _r_ only works with certain return values and not others. Maybe someone else can explain?

[ghoetker]

Debugging an error that you can't recreate is the definition of "above and beyond". Thank you so much!

For general reference, I'm guessing that mu_diff as output from ttest was added after Stata 18, as it is part of the output I get with StataNow 19.5.

. quietly: ttest mpg, by(foreign)

. return list

scalars:
            r(ub_diff) =  -2.230383650411254
            r(lb_diff) =  -7.661224741197126
            r(mu_diff) =  -4.94580419580419

Thankfully, the solution in #3 worked.

This baffles (and frankly frustrates) me as the documentation for collect stars (TABLES, page 161) clearly shows the _r prefix:

Same as above, and attach the stars to coefficients stored in r b
collect stars _r_p 0.01 ”***” 0.05 ”**” 0.1 ”*”, attach(_r_b)

It would be great if someone could explain this discrepancy.

In the meantime, thank you for the working solution.