Content quality | ||||||
---|---|---|---|---|---|---|
% of answers 2 or more points below mean score | % of answers > 1 and < 2 points below mean score | % of answers > 1 and < 2 points above mean score | % of answers 2 or more points above mean score | Bias (score-mean score) (mean ± SD) | P (comparison score with mean score) | |
Rater 1 | 17.6 | 7.4 | 8.3 | 5.6 | -0.45 ± 1.53 | 0.003 |
Rater 2 | 12 | 7.4 | 10.2 | 7.4 | 0.03 ± 1.43 | 0.839 |
Rater 3 | 15.9 | 8.4 | 6.5 | 5.6 | -0.25 ± 1.50 | 0.084 |
Rater 4 | 4.6 | 2.8 | 10.2 | 21.3 | 0.68 ± 1.52 | < 0.001 |
Formal quality | ||||||
% of answers 2 or more points below mean score | % of answers > 1 and < 2 points below mean score | % of answers > 1 and < 2 points above mean score | % of answers 2 or more points above mean score | Bias (score-mean score) (mean ± SD) | P (comparison score with mean score) | |
Rater 1 | 24.1 | 17.6 | 0.9 | 0 | -0.97 ± 1.13 | < 0.001 |
Rater 2 | 2.8 | 3.7 | 2.8 | 0.9 | 0.00 ± 0.71 | 1.000 |
Rater 3 | 9.3 | 9.3 | 4.7 | 0.9 | -0.25 ± 0.98 | 0.012 |
Rater 4 | 1.9 | 0 | 22.2 | 6.5 | 0.66 ± 0.89 | < 0.001 |
Rater 5 | 2.8 | 2.8 | 19.4 | 6.5 | 0.56 ± 0.94 | < 0.001 |