Problems posting and testing ICC estimates

[Jesse Young]

Hi all,

I am running stata 13.1. I am assessing inter-rater reliability (on the same measure with three unique raters) using the ICC command. I cannot seem to save/post the coefficient estimates in order to test if two groups in my sample are significantly different on agreement across raters.

Has anyone encountered this before? Any help is much appreciated. I tried searching the list, however apologies if this has been addressed previously.

Kind Regards,
Jesse.

[Joseph Coveney]

The estimated intraclass correlation coefficients are available as return scalars, as indicated in the command's help file. What have you tried that doesn't work? You might get better assistance if you post the code that you've tried and that fails.

I'm not familiar with icc, having never used it, and I'm not sure that I understand what you want to do. But from reading its help file, it doesn't seem as if icc will do what I'm guessing you're trying to do.

[Jesse Young]

Thanks so much for the reply Joseph.

My code is as follows:

by gender, sort: icc hscore htarget hjudge, consistency

If I then run:

scalar list _all

It gives me nothing (no error message but no list either).

If I then try:

estimates dir (same thing happens)

If I try:

estimates store

I get an error message - last estimation results not found, nothing to store r(301);

What I would like to do is test if the icc estimates are significantly different between males and females (i.e. by gender). Should I just estimate this using the testvalue statement, entering an approximation of the coefficient value? (seems a little crude to me) or is there a way to do this using the actual estimates produced by the icc command?

[Clyde Schechter]

I'm not really familiar with -icc-, but if you check the online help for it, you will see at the end that it stores a lot of results in r(). Those returned results probably include whatever ingredients you need to perform your tests. It's just a matter of collecting them. Since you are doing this using by:, if you just go for the returned values at the end, you will get them only fo rthe final value of the by group (in this case, one of the genders.) So you should also look into -statsby-. Again, the online help should tell you all you need to use this.