Mcnemar & Kappa

[Ana Maria Martinez]

Hi friends

I want to apply Mcnemar to compare ecocardiography 3d & 2D to demonstrate if 3D is superior to 2D. I am also want to apply Kappa to compare both with the gold standard which is the surgical view
These are my data, any help to use the appropiate commands will be welcome

clear
input byte(Lesion PA1 PA1_3D PA1_2D FA1 FA1_3D FA1_2D PA2 PA2_3D PA2_2D FA2 FA2_3D FA2_2D PA3 PA3_3D PA3_2D FA3 FA3_3D FA3_2D PP1 PP1_3D PP1_2D FP1 FP1_3D FP1_2D PP2 PP2_3D PP2_2D FP2 FP2_3D FP2_2D PP3 PP3_3D PP3_2D FP3 FP3_3D FP3_2D)
1 0 0 1 0 0 1 1 1 1 1 1 0 1 1 0 1 1 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 1 1 1 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 
1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 1 1 1 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 1 1 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 1 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 1 1 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 1 1 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 1 0 0 0 0 0 0 0 
1 1 1 0 0 0 0 1 1 0 0 0 0 1 1 1 0 0 0 1 1 1 1 1 0 1 1 1 0 0 0 1 1 1 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 
1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 1 1 1 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 1 1 1 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 1 1 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 
1 1 1 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 1 1 1 0 1 1 1 0 0 0 0 0 0 0 
1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 1 0 0 0 0 0 1 
1 0 0 0 0 0 0 0 1 1 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 1 0 
end
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PA1 Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PA1_3D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PA1_2D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FA1 Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FA1_3D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FA1_2D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PA2 Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PA2_3D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PA2_2D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FA2 Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FA2_3D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FA2_2D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PA3 Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PA3_3D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PA3_2D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FA3 Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FA3_3D Detectado
label def Detectado 0 "No", modify
label values FA3_2D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PP1 Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PP1_3D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PP1_2D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FP1 Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FP1_3D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FP1_2D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PP2 Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PP2_3D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PP2_2D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FP2 Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FP2_3D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FP2_2D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PP3 Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PP3_3D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values PP3_2D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FP3 Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FP3_3D Detectado
label def Detectado 0 "No", modify
label def Detectado 1 "Si", modify
label values FP3_2D Detectado

[Joe Canner]

Ana Maria,

I think we need more information to answer your question. You mention three variables that you want to compare (2D, 3D, and surgery) but you have many more variables than that, with names that do not give much hint as to what they refer to.

As a minor point, it is also not clear why you need to re-define the value label "Detectado" multiple times. The following should be sufficient:

label def Detectado 0 No 1 Si
label values PA1-FP3_2D Detectado

Regards,
Joe

[Robert Picard]

Joe, it looks like Ana used dataex (from SSC) to generate her data example. I did not anticipate the case where multiple variables use the same variable labels and so the labels are redefined for each variable. I'll fix that in the next update.

[Joe Canner]

Robert, that makes sense; I wondered if those lines had been auto-generated.

[Ana Maria Martinez]

Sorry guys, I will post my data in dta to clarify, Thank you for the answers!

[Ana Maria Martinez]

I attach you my data .dta, in summary the mitral valve has 2 leaflets anterior and posterior each with 3 scallops named A1,A2,A3,and P1,P2,P3) and there are two types of lesion with the scallops , prolapse and flail. I want to use Mcnemar to demostrate that Echocardiography3D (Echo3D) is superior to echocardiography2d (2D) and I am also want to use Kappa to compare both with the gold standard which is the detection of the prolapse and flail by the surgeon. Any help would be great!!

[Joe Canner]

Ana Maria,

Thanks, that's helpful. To do a McNemar on one particular measurement (say, prolapse of anterior leaflet scallop #1), you can use the symmetry command, and to do a Kappa, use the kap command:

symmetry PA1_3D PA1_2D, exact

// 3D versus surgery
tab PA1_3D PA1, row col
kap PA1_3D PA1

// 2D versus surgery
tab PA1_2D PA1, row col
kap PA1_2D PA1

I am assuming here that PA1 is the gold standard. The tabulations can also give you sensitivity and specificity relative to the gold standard.

The big question here, I suspect, is how to summarize the 12 different measurements (Flail/Prolapse x Anterior/Posterior x 3 Scallops) and get a single McNemar or Kappa for everything. This is as much a clinical question as a statistical one. You will need to consult the literature to see if and how people typically combine them. For example, can you combine all three scallops into a composite measurement for the entire leaflet? Can anterior and posterior leaflets be combined? Are flail and prolapse typically combined? Keep in mind that if you don't combine them into clinically meaningful composite variables, you either need to report each of the 12 measurements separately or you need to pretend that each of the 12 measurements is an independent observation, which is probably not statistically appropriate. There may be ways to account for the lack of independence but those methods are not available with the basic McNemar and Kappa statistics. Perhaps someone else can comment on whether there are ways to accomplish this with other commands.

Also, it's not clear what the "Lesion" variable is for. Is that important for this analysis?

Regards,
Joe

[Ana Maria Martinez]

Thanks for the answer. You are right the variable lesion is not needed.
On the other hand, I do not need to combine because what I need to proof is if Echo3D is better than Echo2D in each prolapse and flail separately. And also to do kappa for each Echo3D and Echo2D to compare with the fold standard which is the surgeon inspection.
I thought I had to do Mcnemar for each echo3d & echo2d for each lesion

[Joe Canner]

Ana,

I'm not suggesting that you should combine; I just assumed that's what you were looking for because the syntax for McNemar and Kappa is pretty straightforward otherwise. Please let us know if you have any other specific questions that aren't covered by the above answer.

Regards,
Joe