Table 3 Inter-rater reliability results for 47 items in the checklist for implementation (Ch-IMP) (n = 27 reviews)
| Â | Percentage agreement | Kappa (95Â % CI) | AC1 statistic (95Â % CI) |
|---|---|---|---|
Action model | Â | ||
Intervention and service delivery protocols | Â | Â | Â |
 Intervention Heterogeneity | 82 | 0.74 (0.56–0.93) | 0.79 (0.62–0.96) |
Target population | |||
 Age | 85 | 0.80 (0.63–0.98) | 0.83 (0.68–0.98) |
 Gender | 89 | 0.85 (0.68–1.00) | 0.87 (0.74–1.00) |
 Grade | 85 | 0.75 (0.57–0.92) | 0.84 (0.69–0.99) |
 Ethnicity | 82 | 0.74 (0.54–0.94) | 0.79 (0.62–0.96) |
 SES | 74 | 0.62 (0.41–0.84) | 0.71 (0.52–0.90) |
Implementers | |||
 Implementer identified | 100 | 1.00 |  |
 Qualifications | 74 | 0.59 (0.37–0.82) | 0.70 (0.50–0.89) |
 Ethnicity | 96 | 0.84 (0.53–1.00) | 0.96 (0.89–1.00) |
 Age | 96 | 0.82 (0.53–1.00) | 0.96 (0.89–1.00) |
 Gender | 96 | 0.78 (0.37–1.00) | 0.96 (0.89–1.00) |
 Socio-economic status | 100 | 1.00 |  |
 Role of the evaluator | 96 | 0.90 (0.70–1.00) | 0.96 (0.88–1.00) |
Implementing organization | |||
 Leadership | 89 | 0.46 (0.03–0.89) | 0.88 (0.76–1.00) |
 Resourcing | 100 | 1.00 |  |
 Intervention development | 93 | 0.72 (0.34–1.00) | 0.92 (0.82–1.00) |
 Quality of materials | 93 | 0.79 (0.53–1.00) | 0.92 (0.82–1.00) |
 Cultural sensitivity | 93 | 0.71 (0.34–1.00) | 0.92 (0.82–1.00) |
 Training | 82 | 0.64 (0.40–0.88) | 0.80 (0.63–0.96) |
 Program improvement processes | 93 | 0.68 (0.33–1.00) | 0.92 (0.82–1.00) |
 Technical or supervisory guidance | 78 | 0.54 (0.26–0.81) | 0.76 (0.58–0.93) |
Associate organizations and community partners | |||
 Presence/absence of partnership | 93 | 0.81 (0.4 8–1.00) | 0.92 (0.81–1.00) |
 Other partnership proc | 96 | 0.79 (0.45–1.00) | 0.96 (0.89–1.00) |
Ecological context | |||
 Settings considered | 74 | 0.62 (0.42–0.82) | 0.66 (0.46–0.86) |
 # Settings | 48 | 0.37 (0.15–0.59) | 0.40 (0.18–0.62) |
Process evaluation | |||
 Recruitment | 82 | 0.68 (0.46–0.91) | 0.80 (0.63–0.96) |
 Attrition | 74 | 0.63 (0.41–0.85) | 0.71 (0.52–0.90) |
 Minimum attrition | 74 | 0.47 (0.20–0.74) | 0.72 (0.53–0.90) |
 Reach | 93 | 0.82 (0.59–1.00) | 0.92 (0.81–1.00) |
 Minimum reach | 96 | 0.65 (0.02–1.00) | 0.96 (0.89–1.00) |
 Dose delivered | 79 | 0.70 (0.50–0.90) | 0.76 (0.58–0.93) |
 Minimum dose delivered | 85 | 0.45 (0.03–0.86) | 0.85 (0.70–0.99) |
 Dose received | 89 | 0.54 (0.10–0.98) | 0.88 (0.76–1.00) |
 Minimum dose received | 100 | 1.00 |  |
 Fidelity | 78 | 0.60 (0.33–0.87) | 0.76 (0.58–0.93) |
 Minimum fidelity | 100 | 1.00 |  |
 Adaptation | 82 | 0.67 (0.42–0.92) | 0.80 (0.63–0.96) |
 Minimum adaptation | 100 | 1.00 |  |
 Participant engagement | 89 | 0.54 (0.10–0.98) | 0.88 (0.76–1.00) |
 Provider engagement | 93 | 0.64 (0.17–1.00) | 0.92 (0.82–1.00) |
 Co-intervention | 78 | 0.38 (0.03–0.73) | 0.76 (0.59–0.94) |
 Contamination | 74 | 0.42 (0.12–0.71) | 0.72 (0.54–0.90) |
Change model | |||
 Apriori intervention model | 78 | 0.65 (0.40–0.89) | 0.72 (0.52–0.92) |
 Logic diagram used | 100 | 1.00 |  |
Environmment | |||
 Years | 74 | 0.63 (0.41–0.86) | 0.71 (0.52–0.89) |
 Country | 74 | 0.65 (0.45–0.85) | 0.70 (0.51–0.90) |
 Urbanicity | 78 | 0.67 (0.45–0.90) | 0.75 (0.57–0.93) |