Significance levels in Stata

[Alyssa Beavers]

Hello,


I have a question on how Stata calculates significance levels. When I get a P-value of 0.000, what is this equivalent to? In other words, is it P<0.001? Is it P<0.0001? P<0.0005? I'm not sure where the rounding occurs, so I don't know what the correct way to report this is.

Many thanks,
Alyssa

[William Lisowski]

That means the P-value when presented with three significant digits is 0.000, so it will be a value less than .0005.

. display %9.3f 0.0005 %9.3f 0.00049999999
    0.001    0.000

[Nick Cox]

If you care -- and there can be good reasons to care -- then you can usually see more decimal places. Sometimes the P-value is a returned result. If it isn't, then start with advice in


SJ-14-1 st0332 . . . . . . Stata tip 116: Where did my p-values go? (Part 3)
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M. L. Buis
Q1/14 SJ 14(1):218--220 (no commands)
discusses how to recover the standard errors for back-
transformed parameters (standard deviation and correlation)

SJ-12-4 st0280 . . . . . . Stata tip 112: Where did my p-values go? (Part 2)
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M. L. Buis
Q4/12 SJ 12(4):759--760 (no commands)
shows how p-values can be recovered for other tests that
are sometimes displayed by estimation commands

SJ-7-4 st0137 . . . . . . . . . . . Stata tip 53: Where did my p-values go?
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M. L. Buis
Q4/07 SJ 7(4):584--586 (no commands)
tip on calculating the t statistic, p-values, and
confidence intervals using returned results

[Joro Kolev]

Reading relevant literature never hurts. But calculating the exact p-value is trivial, and once you have it, you can manipulate your display format anyways you want. Here:

. sysuse auto, clear
(1978 Automobile Data)

. reg price mpg head

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(2, 71)        =     10.44
       Model |   144280501         2  72140250.4   Prob > F        =    0.0001
    Residual |   490784895        71  6912463.32   R-squared       =    0.2272
-------------+----------------------------------   Adj R-squared   =    0.2054
       Total |   635065396        73  8699525.97   Root MSE        =    2629.2

------------------------------------------------------------------------------
       price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         mpg |  -259.1057   58.42485    -4.43   0.000    -375.6015   -142.6098
    headroom |  -334.0215   399.5499    -0.84   0.406    -1130.701    462.6585
       _cons |   12683.31   2074.497     6.11   0.000     8546.885    16819.74
------------------------------------------------------------------------------

. dis  2*ttail(e(df_r),abs(_b[mpg]/_se[mpg]))
.00003291

. dis  2*ttail(e(df_r),abs(_b[head]/_se[head]))
.4059628

[Andrew Musau]

In scientific publications, I have never seen anyone reporting p<0.0005. Examples of common presentations include p=0.01, p=0.015, p=0.001 and for anything lower than 0.0005, p<0.001.

[Nick Cox]

Andrew Musau In genomics etc. there is often interest in and concern for much, much smaller P-values. All that just goes to show how statistical science is far from homogeneous,

[Andrew Musau]

Thanks for the information Nick Cox, this shows how little I know of the conventions outside Economics!

[Mike Lacy]

It's also worth noting for Alyssa that listing p-values that are less than (say) 0.0005 as 0.000 is by no means a peculiarity of Stata. I've seen it or something similar in every other data analysis program I've ever used. I'd rather see such things default to scientific notation, but that default is not common.

[Alyssa Beavers]

Hi all,

Thanks for your responses. Just to clarify the reason I'm asking: in my field we don't really care if a p-value is less than 0.001, I just wanted to make sure that what I report is a correct interpretation of my Stata output.

-Alyssa