Table 3 Inter-rater agreement for binary ratings of metrics using Cohen’s Kappa, Gwet’s AC1, and observed agreement
Study design feature | Cohen’s Kappa | 98.3% CI | Gwet’s AC1 Coefficient | 98.3% CI | Observed agreement |
|---|---|---|---|---|---|
Power analysis | 0.58 | 0.13, 1.00 | 0.94 | 0.88, 1.00 | 0.95 |
Randomization | 0.85 | 0.72, 0.99 | 0.85 | 0.72, 0.99 | 0.93 |
Blinding | 0.79 | 0.62, 0.95 | 0.82 | 0.67, 0.97 | 0.90 |