Skip to main content

Table 9 Levels and abbreviations of the factors constituting the simulation space, inter-rater agreement measures and weights

From: The impact of grey zones on the accuracy of agreement measures for ordinal tables

R×R table

Abbr.

Level of true agreement

Abbr.

Table structure

Abbr.

q=3

3×3

Low

L

Balanced

BL

q=4

4×4

Medium

M

Slightly Unbalanced

UB1

q=5

5×5

High

H

Heavily Unbalanced

UB2

q=6

6×6

    

Sample size

Abbr.

Inter-rater agreement measure

Abbr.

Type of weight

Abbr.

50

n=50

Cohen’s kappa

κ

Unweighted

unweighted

100

n=100

Scott’s π

Ï€

Linear weight

linear

200

n=200

Gwet’s AC2

AC2

Quadratic weight

quadratic

500

n=500

Brennan - Prediger

BP

Ordinal weight

ordinal

1000

n=1000

Krippendorff’s α

α

Radical weight

radical