How to get the p-value from the concord command and how to decide between concord and icc?

[Laza Razafimbelo]

Dear all,

I have some variables collected twice by two groups of surveyors to check the reliability of the answers given by the respondents. To do so, I am using the concord and the icc commands from SSC in Stata 14.2 and have 2 questions. I would be so grateful if someone can help.

1) from concord command I would like to store the p-value of the rho_c but I get a different figure when trying to re-calculate and store it.
Here is the concord code:

. concord aqnty_hrvs_3_1 mqnty_hrvs_3_1

 Concordance correlation coefficient (Lin, 1989, 2000):

 rho_c   SE(rho_c)   Obs    [   95% CI   ]     P        CI type
---------------------------------------------------------------
 0.768     0.131       9     0.512  1.025    0.000   asymptotic
                             0.371  0.928    0.001  z-transform

Pearson's r =  0.833  Pr(r = 0) = 0.005  C_b = rho_c/r =  0.923
Reduced major axis:   Slope =     0.717   Intercept =    -0.326

Difference = aqnty_hrvs_3_1 - mqnty_hrvs_3_1

        Difference                 95% Limits Of Agreement
   Average     Std Dev.             (Bland & Altman, 1986)
---------------------------------------------------------------
    -1.333       4.000                 -9.173      6.507

Correlation between difference and mean = -0.523

Bradley-Blackwood F = 1.918 (P = 0.21670)

Here I am trying to store the p-value and where I get a different figure:

. local pval_areg=2*ttail(e(df_r), abs(`r(rho_c)'/`r(se_rho_c)'))
. di %5.4f `pval_areg'
0.0006

So I get Pr(r = 0) = 0.005 from the first table and 0.0006 from the calculation. Can someone help to tell me what I am doing wrong?

2) as I mentioned earlier, along with the previous concord command I am also using the icc command. The two commands give similar coefficients r(rho_c)=0.768 and r(icc_i)=0.788 but their interpretations are very different.

For r(rho_c)=0.768 we interpret that correcaltion as 'poor' but for r(icc_i)=0.788 'good'.
Which one is right? Is one of them not relevant for what I want to do?

Here is the code for the icc and the reshape of the dataset:

. keep aqnty_hrvs_3_1  mqnty_hrvs_3_1
. rename aqnty_hrvs_3_1 mqnty_hrvs_3_2
. gen num=_n
. reshape long mqnty_hrvs_3_, i(num) j(j)
. icc mqnty_hrvs_3_ num j

Intraclass correlations
Two-way random-effects model
Absolute agreement

Random effects: num              Number of targets =         9
Random effects: j                Number of raters  =         2

--------------------------------------------------------------
         mqnty_hrvs_3_ |        ICC       [95% Conf. Interval]
-----------------------+--------------------------------------
            Individual |   .7887014       .3467217    .9472216
               Average |   .8818704       .5149122    .9728955
--------------------------------------------------------------
F test that
  ICC=0.00: F(8.0, 8.0) = 8.47                Prob > F = 0.003

Note: ICCs estimate correlations between individual measurements
      and between average measurements made on the same target.

Thanks in advance,
Laza

[Nick Cox]

concord was most recently updated via the Stata Journal website.

I think you'll have to post your data before I can add any precise check on your P-value calculation.

Putting words to correlations always strikes me as fairly silly. (If I appear to have written otherwise, that's an illusion.) 0.9 could be disastrous or beyond reach in different contexts. Is 1.8 m tall for an adult male? Not for basketball! The standard for what is good is what is good in your field with your kind of data for your purpose.