Kappa Test student against gold standard

[Rodrigo Badilla]

Hi all,

I need compare diagnostic from 5 students in 50 patient against the right diagnostic, I only found the command:

kap student1 student2 student3...

But this only compare bettween students but not each student against "gold standard".

¨Please any comment could help me.

Regards
Rodrigo

[Clyde Schechter]

The command to do what you seek will depend in part on how your data is laid out. So you need to show us a sample of that.

[Rodrigo Badilla]

My data looks like:

input subject gold student1 student2 student3

1 1 0 1 1
2 1 1 0 0
3 0 1 1 1
4 1 1 1 1
5 0 1 1 0
6 0 1 1 0
7 0 1 1 1
end

[Clyde Schechter]

So for this configuration, you need to loop over the students:

forvalues j = 1/3 {
     display "Kappa for student `j'"
     kap gold student`j'
}

That said, some would argue that you should not be doing a kappa statistic when one of the "ratings" is in fact the correct diagnosis. Conceptually it is more appropriate to calculate the sensitivity and specificity of each student's diagnoses in that setting. (Others would argue that the "correct" diagnosis is not guaranteed to be correct in real life and is really just another rating, so that agreement in the symmetric sense provided by Cohen's kappa remains appropriate.)

forvalues j = 1/3 {
     display "Operating characteristics for student `j'"
     tab student`j' gold, col
}

For this the specificity and sensitivity are found as the column percentages in the upper and lower diagonal cells, respectively.

[Rodrigo Badilla]

Thanks Clyde for you answer. I will run the comman for specificity and sensitivity, to complement my results.

Thanks and Regards.
Rodrigo

[Nick Cox]

If your ratings are just 0s and 1s there is probably a more direct approach than using kappa at all.

[Rodrigo Badilla]

yes are 0s and 1s.

[Rodrigo Badilla]

Dear all,

Do you know if exist a method to compare all the students against gold standard??. I would like compare kappa from this student generattion with previous generation.
Regards