Skip to main content

Table 4 Category-specific and overall inter-rater reliability coefficients, sensitivity and specificity

From: Examining intra-rater and inter-rater response agreement: A medical chart abstraction study of a community-based asthma care program

 

Percentage agreement

Kappa

Sensitivity§

Specificity§

Category (number of items)

Category %

% range across items

Category κ (95% CI)

κ range across items

Estimate (95% CI)

Estimate (95% CI)

1. Physical assessment (8)

95

79–100

0.88 (0.87–0.90)

0.00–1.00

0.95 (0.92–0.97)

0.95 (0.91–0.98)

2. Asthma control (7)

84

76–93

0.68 (0.65–0.70)

0.05–0.71

0.89 (0.84–0.92)

0.85 (0.80–0.88)

3. Spirometry (6)

100

100

nc

nc

nc

0.99 (0.98–1.00)

4. Asthma education (8)

76

54–95

0.49 (0.46–0.52)

0.04–0.67

0.87 (0.80–0.92)

0.77 (0.73–0.81)

5. Asthma referrals (3)

90

88–93

0.77 (0.73–0.81)

0.02–0.84

0.88 (0.77–0.95)

0.95 (0.90–0.98)

6. Medications (1)

90

90

nc

nc

nc

0.94 (0.85–0.98)

Overall

88

 

0.75* (0.75–0.77)

 

0.91 (0.89–0.93)

0.89 (0.88–0.91)

  1. * Chi-square test statistic for homogeneity = 737, df = 4, p < 0.0001
  2. § Sensitivity and specificity are calculated to compare assessments of all raters against the gold standard.
  3. Abbreviations: κ kappa statistic, CI confidence interval, nc not calculable