Measuring inter-rater reliability for nominal data – which coefficients and confidence intervals are appropriate?

Table 2 Results of the case study (n = 50) of histopathological assessment of patients with mamma carcinoma rated by four independent and blinded readers. The six ordinal parameters were also assessed if as they were measured in a nominal way

Parameter	Levels	Scale	Missing values (in %)	Observed agreement	Fleiss’ K			Krippendorff’s alpha
Parameter	Levels	Scale	Missing values (in %)	Observed agreement	Point estimate	Asymptotic CI	Bootstrap CI	Point estimate	Bootstrap CI
Estrogen IRS	2	Nominal	0	96 %	0.88	0.76–0.99	0.65–1.00	0.88	0.66–1.00
MIB-1 status	2	Nominal	0	72 %	0.66	0.55–0.78	0.51–0.80	0.66	0.51–0.80
HER-2 status	3	Nominal	0	86 %	0.77	0.68–0.87	0.58–0.90	0.77	0.60–0.92
Estrogen intensity	4	Nominal	0	78 %	0.62	0.54–0.71	0.42–0.78	0.62	0.40–0.79
Estrogen intensity	4	Ordinal	0	78 %	-	-	-	0.74	0.51–0.80
Estrogen group	5	Nominal	0	86 %	0.74	0.66–0.82	0.55–0.88	0.74	0.55–0.89
Estrogen group	5	Ordinal	0	86 %	-		-	0.88	0.73–0.96
Progesteron intensity	4	Nominal	10	77 %	0.74	0.63–0.84	0.56–0.89	0.69	0.53–0.83
Progesteron intensity	4	Ordinal	10	77 %	-	-	-	0.86	0.75–0.93
Progesteron group	5	Nominal	0	44 %	0.56	0.50–0.63	0.43–0.66	0.56	0.45–0.67
Progesteron group	5	Ordinal	0	44 %	-	-	-	0.83	0.72–0.90
HER-2 score	4	Nominal	0	46 %	0.52	0.45–0.60	0.38–0.64	0.52	0.37–0.65
HER-2 score	4	Ordinal	0	46 %	-	-	-	0.70	0.53–0.82
MIB-1 proliferation rate	10	Nominal	0	10 %	0.20	0.15–0.25	0.12–0.28	0.20	0.12–0.27
MIB-1 proliferation rate	10	Ordinal	0	10 %	-	-	-	0.81	0.68–0.87

ISSN: 1471-2288