Skip to main content

Table 7 Reported inter-rater reliability of GAS in included studies

From: A systematic review to investigate the measurement properties of goal attainment scaling, towards use in drug trials

First author Year Drug study N Methods and results Quality
Bovend’Eert 2011 No 29 Mixed model ICC(a, k) between therapist and masked assessor scoring procedures is 0.478 (low); LoA −1.52 +/− 24.54. -
ICC ≤0.7
Brown 1998 No 24 The Pearson’s r correlations and inter-rater ICCs (2,1) between the scores of the treating therapist and the independent raters were r = 0.84 (p <0.0001, n = 360, r2 = 70.90/0) and ICC = 1.00 (between raters: (IF = 1, SS = 0.01; within raters: df = 695, SS = 1, 172.65), respectively. The coefficients between scores of the 2 independent raters were r = 0.81 (p <0.0001, n = 135, rZ = 66.2 %) and ICC = 0.997 (between raters: dl = 1, SS = 1.48; within raters: f = 245, SS = 433,39). The results support acceptable inter-rater reliability of the scores for the goals in this study. +
ICC ≥0.7
De Beurs 1993 Yes 40 Agreement on the content of the chosen goals was measured between the intakers, in other words the people who performed the first session before the patients were randomized, and therapists was measured. Also, the agreement between the therapists and the people who objectively set the goals, or the goal setters, was measured. Agreement between goal setters and therapists and between goal setters and intakers was 43 and 57 % respectively. The calculations used to establish the agreement were not reported. -
Unclear design or method, agreement ≤0.7
Palisano 1993 No 21 Before data collection, an inter rater reliability was measured between the author and an examiner (Kappa = 0.89, agreement 90 %). During the study 16 goals were simultaneously scored. The agreement was 88 % (Kappa = 0.75). +
ICC ≥0.7
Rockwood 1993 No 45 A primary nurse and a multidisciplinary team scored GAS, ICC = 0.91. +
ICC ≥0.7
Rockwood 1997 No 44 ICC = 0.95 for admission scoring, ICC = 0.95 for discharge scoring, ICC = 0.93 for change score. +
ICC ≥0.7
Ruble 2012 No 35 + 44 (reference to previous study) Two raters independently coded 20 % of the GAS forms for the three features of agreement in sample 1 and 2. ICC for average agreement in sample 1 on measurability (0.96, 95 % CI [.87, .99]), difficulty (0.59, 95 % CI [−.18, .81]) and equidistance (0.96, 95 % CI [.74, .99); ICC for average agreement in sample 2 on measurability (1.0), difficulty (0.96, 95 % CI [.83, .99]) and equidistance (0.96, 95 % CI [.84, .99]). +
Only ICC for difficulty is lower than 0.7
Ruble 2013-a No 49 Two coders independently coded 39 % of the goals, ICC for social skills = 0.82, ICC for communication skills = 0.86, ICC for learning skills = 0.91. +
ICC ≥0.7
Ruble 2013-b No Not stated (reference to previous study) Excellent inter-rater reliability was achieved for both study 1 (ICC = 0.99) and study 2 (ICC = 0.90). +
ICC ≥0.7
Steenbeek 2005 Yes 11 A video scoring and scoring by a physiotherapist were compared, gaining a Kappa of 0.63. 5 out of 33 of the goal scores differed significantly (tested with a Wilcoxon signed rank test). -
k ≤0.7
Stolee 1999 No 173 ICC (N = 61) = 0.93 of GAS follow-up score. ICC (N = 61) = 0.89 of the separate goals, when checked whether the goals have been attained. +
ICC ≥0.7
Woodward 1978 No 279 Correlation of two goal attainment scores: 0.84. 33 % scored identical, 78 % within one level, 95 % within two levels. GAS scores did not differ significantly (F(6,268) = 1.25, P >0.10). -
Non-standard way of measuring inter-rater reliability