Table 4 Mean of errors (m_e) / distance between index estimations and targets of estimation

Interrater Reliability

2

Interrater Reliability Estimator

a_o

S

I_r

AC₁

π

κ

α

3

m_e (r_i) = mean (|r_i-o_ri|) (0 ≤ m_e ≤ 1)

.130^***

.096^***

.180^***

.093^***

.327^***

.324^***

.323^***

4

Standard Deviation of m_e (r_i)

.145

.099

.148

.104

.221

.220

.220

5

95% confidence interval of m_e (r_i)

.115 ~ .144

.086 ~ .106

.164 ~ .194

.082 ~ .103

.304 ~ .349

.302 ~ .346

.301 ~ .345

Chance Agreement

6

Chance Agreement Estimator

ao_ac

S_ac

Ir_ac

AC_ac

π_ac

κ_ac

α_ac

7

m_e (a_c):=mean (|a_c-o_ac|) (0 ≤ m_e ≤ 1)

.130^***

.182^***

.182^***

.130^***

.450^***

.448^***

.448^***

8

Standard Deviation of m_e (a_c)

.145

.141

.141

.127

.201

.201

.202

9

95% confidence interval of m_e (a_c)

.115 ~ .144

.168 ~ .196

.168 ~ .196

.117 ~ .143

.429 ~ .470

.428 ~ .469

.427 ~ .468

N

10

N_c (number of rating sessions)

384

384

384

384

384

384

384

11

N_d (number items within each session)

100

100

100

100

100

100

100