Observer agreement paradoxes in 2x2 tables: comparison of agreement measures

Table 2 Scenarios studied in this manuscript: Cell frequencies, marginals, proportion observed, bias and prevalence index

Scenario	Type of table	x₁₁	x₁₂	x₂₁	x₂₂	f1	f2	g1	g2	P_O	BI	PI
Paradox 1:
1	Symmetrical balance	40	9	6	45	49	51	46	54	.85	.03	-.05
2	Symmetrical imbalance	80	10	5	5	90	10	85	15	.85	.05	.75
3	Perfect symmetrical imbalance	90	5	5	0	95	5	95	5	.90	0	.90
Paradox 2:	P _O set at 0.60
4	Symmetrical imbalance	45	15	25	15	60	40	70	30	.60	-.10	.30
5	Asymmetrical imbalance	25	35	5	35	60	40	30	70	.60	.30	.10
6	Perfect symmetrical imbalance	40	20	20	20	60	40	60	40	.60	0	.20
7	Asymmetrical imbalance	40	35	5	20	75	25	45	55	.60	.30	.20
8	Asymmetrical imbalance	30	30	10	30	40	60	60	40	.60	.20	0
	P _O set at 0.90
9	Perfect symmetrical imbalance	85	5	5	5	90	10	90	10	.90	0	.80
10	Symmetrical imbalance	70	10	0	20	80	20	70	30	.90	.10	.50
	P _O low (≤50%)
11	Perfect symmetrical balance	25	25	25	25	50	50	50	50	.50	0	0
12	Asymmetrical imbalance	30	30	20	20	60	40	50	50	.50	.10	.10
13	Perfect symmetrical balance	20	30	30	20	50	50	50	50	.40	0	0
14	Perfect symmetrical balance	5	45	45	5	50	50	50	50	.10	0	0

ISSN: 1471-2288