ICC between three raters with continuous variables

[Mike Woods]

Hi all,

I am trying to do an ICC between three raters on continuous variables (x-ray measurement readings) to get inter-observer agreement statistics. Is there a way to do this in Stata as the icc command currently only allows me to analyze between two raters? If not, is there another method of inter-observer analysis I should do in this case? Would greatly appreciate help and can provide more specific information if needed.

Thank you!

[Joseph Coveney]

. . . the icc command currently only allows me to analyze between two raters

Where do you get that?

The illustrative example shown at the bottom of the help file for icc has four raters.

webuse judges
tabulate judges

[Mike Woods]

Where do you get that?

The illustrative example shown at the bottom of the help file for icc has four raters.

webuse judges
tabulate judges

Sorry, I'm very new to Stata. Whenever I use the ICC command with multiple variables after that, it doesn't work and says it doesn't allow that many inputs and only works when I try two inputs? Is there something I am doing wrong? Ex code: icc rater1 rater2 rater3

[daniel klein]

icc requires data in long-format, meaning that ratings are recorded in one variable, subjects (x-rays) in another variable and raters are identified by a third variable. You probably have ratings of each rater recorded in a separate variable where observations represent subjects. You need to reshape your data to look like judges.dta.

Edit: Soemthing like:

generate double xray = _n
reshape long rater , i(xray) j(rater_id)
rename rater reading
icc reading xray rater_id

Alternatively, download kappetc from SSC; it will work directly with your current setup as something like

ssc install kappaetc
kappaetc rater1 rater2 rater3 , icc(model)

where model is one of oneway, random, or mixed. For more information, read

help kappaetc icc

and the references therein.

[Joseph Coveney]

Is there something I am doing wrong? Ex code: icc rater1 rater2 rater3

Yeah, you need to reshape long your data so that the three raters are in one variable.

Try something along the following lines.

generate long target = _n
rename rater? score?
reshape long score, i(target) j(rater)
icc score target rater

The code above assumes that your dataset doesn't already have a variable identifying the targets, which I assume are patients whose radiographs the three raters are evaluating.