Skip to main content

Table 4 Tendency of the individual raters to score above or below the mean score

From: Quality assessment of expert answers to lay questions about cystic fibrosis from various language zones in Europe: the ECORN-CF project

Content quality
  % of answers 2 or more points below mean score % of answers > 1 and < 2 points below mean score % of answers > 1 and < 2 points above mean score % of answers 2 or more points above mean score Bias (score-mean score) (mean ± SD) P (comparison score with mean score)
Rater 1 17.6 7.4 8.3 5.6 -0.45 ± 1.53 0.003
Rater 2 12 7.4 10.2 7.4 0.03 ± 1.43 0.839
Rater 3 15.9 8.4 6.5 5.6 -0.25 ± 1.50 0.084
Rater 4 4.6 2.8 10.2 21.3 0.68 ± 1.52 < 0.001
Formal quality
  % of answers 2 or more points below mean score % of answers > 1 and < 2 points below mean score % of answers > 1 and < 2 points above mean score % of answers 2 or more points above mean score Bias (score-mean score) (mean ± SD) P (comparison score with mean score)
Rater 1 24.1 17.6 0.9 0 -0.97 ± 1.13 < 0.001
Rater 2 2.8 3.7 2.8 0.9 0.00 ± 0.71 1.000
Rater 3 9.3 9.3 4.7 0.9 -0.25 ± 0.98 0.012
Rater 4 1.9 0 22.2 6.5 0.66 ± 0.89 < 0.001
Rater 5 2.8 2.8 19.4 6.5 0.56 ± 0.94 < 0.001
  1. The numbers represent the percentage of expert answers that were scored more than one respectively two points above/below the mean score from all raters. Rater 5 as a representative of the German CF-patient organization and not a care team member scored only the formal aspect of the expert answers. To assess the bias of raters, the mean difference between the individual score given by a rater and the mean score of the group of raters was calculated. A p value of p < 0.05 was regarded to determine statistically significant differences between individual scores and the mean score.
\