Skip to main content

Table 4 Criteria-level intraclass correlation coefficient and 95% confidence interval by year.

From: Reviewer agreement trends from four years of electronic submissions of conference abstract

  2001 2002 2003 2004
Criteria ICC 95% CI ICC 95% CI ICC 95% CI ICC 95% CI
Hypothesis 0.24 0.11–0.37 0.27 0.16–0.39 0.25 0.13–0.37 0.26 0.16–0.36
Study Design 0.15 0.03–0.29 0.20 0.09–0.32 0.30 0.18–0.42 0.23 0.13–0.33
Methods I 0.26 0.14–0.40 0.27 0.16–0.39 0.57 0.47–0.66 0.44 0.35–0.53
Methods II   -   - 0.59 0.50–0.68 0.51 0.43–0.60
Statistics 0.34 0.22–0.47 0.38 0.27–0.50 0.43 0.32–0.54 0.41 0.31–0.50
Presentation 0.16 0.04–0.29 0.17 0.06–0.28 0.34 0.22–0.46 0.17 0.08–0.27
Originality   -   - 0.25 0.14–0.37 0.20 0.10–0.30
Impact 0.22 0.10–0.36 0.24 0.12–0.36 0.34 0.22–0.46 0.32 0.22–0.42
Overall 0.31 0.18–0.44 0.30 0.19–0.42 0.50 0.39–0.60 0.30 0.20–0.40
Conclusion 0.24 0.11–0.37 0.18 0.07–0.29     
Recommendation 0.21 0.09–0.35 0.34 0.23–0.45     
  1. NB: Bold highlights ICC values that are greater than 0.40 ("moderate agreement")