Skip to main content

Table 5 Coverage rates of common γ 95% confidence intervals of the three proposed methods based on 10,000 simulations

From: Homogeneity score test of AC1 statistics and estimation of common AC1 in multiple or stratified inter-rater agreement studies

Balanced π conditions

Unbalanced π conditions

n1 = n2

γ0

π1 = π2

SA

FZ

PV

n1 = n2

γ0

π1

π2

SA

FZ

PV

20

0.1

0.5

0.939

0.959

0.958

20

0.1

0.5

0.35

0.939

0.958

0.958

0.3

0.936

0.958

0.950

0.3

0.933

0.958

0.955

0.5

0.931

0.962

0.962

0.5

0.927

0.959

0.960

0.7

0.924

0.976

0.963

0.7

0.917

0.971

0.961

0.9

0.998

0.963

0.955

0.9

0.997

0.966

0.955

0.1

0.35

0.935

0.953

0.956

0.1

0.65

0.35

0.938

0.955

0.957

0.3

0.934

0.955

0.953

0.3

0.934

0.955

0.956

0.5

0.926

0.956

0.955

0.5

0.927

0.959

0.955

0.7

0.918

0.965

0.958

0.7

0.917

0.967

0.960

0.9

0.996

0.967

0.947

0.9

0.996

0.969

0.950

0.7

0.2

0.929

0.959

0.950

0.7

0.5

0.2

0.931

0.965

0.956

0.9

0.970

0.966

0.911

0.9

0.993

0.970

0.943

50

0.1

0.5

0.949

0.953

0.952

50

0.1

0.5

0.35

0.946

0.952

0.953

0.3

0.945

0.955

0.954

0.3

0.943

0.952

0.951

0.5

0.945

0.954

0.953

0.5

0.941

0.952

0.952

0.7

0.936

0.961

0.953

0.7

0.936

0.956

0.953

0.9

0.920

0.971

0.971

0.9

0.923

0.968

0.965

0.1

0.35

0.942

0.948

0.952

0.1

0.65

0.35

0.946

0.954

0.954

0.3

0.942

0.950

0.950

0.3

0.943

0.953

0.952

0.5

0.940

0.951

0.951

0.5

0.941

0.954

0.953

0.7

0.937

0.954

0.952

0.7

0.936

0.956

0.953

0.9

0.926

0.966

0.960

0.9

0.925

0.968

0.960

0.7

0.2

0.938

0.949

0.948

0.7

0.5

0.2

0.937

0.954

0.952

0.9

0.927

0.965

0.954

0.9

0.928

0.969

0.960

80

0.1

0.5

0.945

0.952

0.952

80

0.1

0.5

0.35

0.946

0.951

0.951

0.3

0.947

0.952

0.952

0.3

0.946

0.952

0.952

0.5

0.943

0.953

0.952

0.5

0.942

0.950

0.950

0.7

0.944

0.950

0.949

0.7

0.943

0.955

0.953

0.9

0.901

0.962

0.956

0.9

0.922

0.963

0.957

0.1

0.35

0.946

0.951

0.951

0.1

0.65

0.35

0.943

0.947

0.946

0.3

0.944

0.949

0.949

0.3

0.945

0.950

0.950

0.5

0.944

0.950

0.950

0.5

0.944

0.953

0.952

0.7

0.941

0.950

0.949

0.7

0.939

0.953

0.952

0.9

0.931

0.960

0.953

0.9

0.927

0.963

0.954

0.7

0.2

0.944

0.950

0.949

0.7

0.5

0.2

0.942

0.955

0.954

0.9

0.929

0.956

0.951

0.9

0.927

0.961

0.955

  1. SA, FZ, and PV refer to 95% confidence intervals for common AC1 using the simple asymptotic, Fisher’s Z transformation, and profile variance methods, respectively