Table 9 Levels and abbreviations of the factors constituting the simulation space, inter-rater agreement measures and weights
From: The impact of grey zones on the accuracy of agreement measures for ordinal tables
R×R table | Abbr. | Level of true agreement | Abbr. | Table structure | Abbr. |
|---|---|---|---|---|---|
q=3 | 3×3 | Low | L | Balanced | BL |
q=4 | 4×4 | Medium | M | Slightly Unbalanced | UB1 |
q=5 | 5×5 | High | H | Heavily Unbalanced | UB2 |
q=6 | 6×6 | ||||
Sample size | Abbr. | Inter-rater agreement measure | Abbr. | Type of weight | Abbr. |
50 | n=50 | Cohen’s kappa | κ | Unweighted | unweighted |
100 | n=100 | Scott’s π | π | Linear weight | linear |
200 | n=200 | Gwet’s AC2 | AC2 | Quadratic weight | quadratic |
500 | n=500 | Brennan - Prediger | BP | Ordinal weight | ordinal |
1000 | n=1000 | Krippendorff’s α | α | Radical weight | radical |