Skip to main content

Table 1 Domains and items within each domain for the Checklist for Implementation (Ch-IMP)

From: Development, inter-rater reliability and feasibility of a checklist to assess implementation (Ch-IMP) in systematic reviews: the case of provider-based prevention and treatment programs targeting children and youth

Action model
 Intervention & service delivery protocol Intervention heterogeneity Whether consideration was given to the range of strategies, elements, activities, types of components for the interventions included within the review. If relevant, please use the comment function to elaborate on the specific aspects of intervention heterogeneity captured in the review.
 Target population Age Specific age or age range of participants
Gender Gender of participants
Grade Specific grade or grade range of participants
Ethnicity Ethnic background of participants
Socio-economic status Income, highest level of education, occupation of caregivers of participants
 Implementers Implementer identifieda Identify who implements the intervention and interfaces with participants.
Qualifications Consideration to different types of implementers; please consider whether reviews considered implementer’s education level, certifications, or past relevant experiences to assess their ability to do the job.
Ethnicity Ethnic background of the implementers.
Age Specific age or age range of implementers.
Gender Gender of implementers.
SES One or more of implementer’s income, highest level of education or occupation.
Role of evaluator Whether role of the evaluator was addressed. (i.e., roles in program delivery vs evaluation).
 Implementing organization Leadership Whether program champions and leaders provide instructions or guidance to staff/ implementers to facilitate the intervention delivery.
Resourcing Resources includes having sufficient personnel/ staff, financial resources/ operational budget, space, buildings or sites (physical resources), and materials/ equipment (technological resources) to run the program.
Intervention development Intervention development can be strengthened through strategic program planning and program design processes including intervention mapping, needs assessment, pilot-testing, formative evaluation, evaluability assessment or other developmental work.
Quality of materials The quality of materials is commonly assessed in relation to the quality of the intervention materials (e.g., activity materials, curriculum) or the training materials/manual.
Cultural sensitivity Interventions that consider the language, socio-cultural values and traditions may be considered more appropriate to the cultural groups in which they are intended to benefit.
Training Assess whether any consideration has been given to training, the quality of training or any other aspect of training that acts to enhance the skills/ competency of service delivery staff.
Program improvement processes Information from intervention improvement processes such as performance monitoring, feedback, formative evaluation, intervention monitoring can improve delivery.
Technical or supervisory guidance Providing implementers with practical or expert support and guidance (unrelated to intervention improvement processes) during their implementation efforts to improve implementation quality.
 Associate organizations & community partners Presence/absence of partnership Note any formal partnerships or collaborations during intervention planning or implementation
Other partnership processes Note one or more aspects of the collaboration or partnership such as pooling resources, dividing responsibilities for different aspects of complex intervention delivery.
 Ecological context Settings considered Please specify whether this review formally considered the setting in which the intervention was implemented.
Settingsb Please specify whether the number of settings in which the interventions were implemented.
Program implementation (process evalution)
 Recruitment Refers to specific information on the procedures used to recruit participants into or attract participants to the intervention.
 Attrition Attrition is a measure of drop-out rates, or the proportion of participants lost during the course of an intervention or during follow up.
 Minimum attrition Please determine whether the review considered a minimum attrition/drop-out rate.
 Reach Reach refers to the degree to which the intended audience participates in an intervention by ‘their presence’.
 Minimum reach Please determine whether the review considered implementation of minimum reach.
 Dose delivered Dose delivered is established through the efforts and actions of implementers or implementing organization. This concept refers to the proportion or amount of an intervention (or the combined strategies) delivered to participants; often measured through frequency (e.g., twice per week), duration (e.g., duration of program in months) and intensity (e.g., total a program delivery hours).
 Minimum dose delivered Please determine whether the review considered implementation of minimum dose delivered.
 Dose received Dose received, also referred to as exposure, is a characteristic of the target populations’ engagement and active participation in an intervention. It is an objective measure of the extent to which participants actually utilise and interact with program strategies, materials, or resources.
 Minimum dose received Please determine whether the review considered implementation of minimum dose received.
 Fidelity Intervention fidelity is a commonly used measure in process evaluation. It has been conceptualised and measured in a variety of ways. Its essential definition reflects the extent to which an intervention is implemented as originally intended by program developers. It has been applied to assessing intervention strategies to the integrity of an implementing system (i.e., “the extent to which an intervention has been implemented as intended by those responsible for its development”; “closeness between the program-as-planned and the program-as-delivered”; “faithful replication”; the degree to which “specified procedures are implemented as planned”). Please use the comment function to provide the definition used in the review.
 Minimum fidelity Please determine whether the review considered implementation of minimum fidelity.
 Adaptation The extent to which program content is intentionally or purposefully changed during implementation, from the original standard, to enhance program effectiveness. Programs can be adapted to be situationally responsive to local needs and circumstances. Please note the reasons for adaptation.
 Minimum adaptation Please determine whether the review considered implementation of minimum adaptation.
 Participant engagement Refers to the subjective attributes that define their participation in, interaction with or receptivity to an intervention. This can include what they think of the program (cognitive orientation) such as satisfaction with the program, commitment, perceived relevance of the program of the outcomes or how they feel about the program (affective orientation) such as enthusiasm or enjoyment.
 Provider engagement Implementer engagement refers to the subjective attributes of program staff that can influence their capacity to deliver intervention strategies. This can include: a) what staff think about the program content (cognitive orientation) such as satisfaction with the program, commitment, motivation, perceived importance/ buy-in, perceived relevance of the program of the outcomes; b) how staff feel when implementing the program (affective orientation) such as enthusiasm or enjoyment; or c) staff’s interpersonal style or the methods used to communicate concepts (e.g., warmth, empathy).
 Co-intervention When interventions other than the treatment under study are applied differently to the treatment and control/comparison groups.
 Contamination When an intervention is unintentionally delivered to participants in the control group or inadvertent failure to deliver the intervention to the experimental group.
Change model
 A priori change modelc The Change Model links intervention strategies to a sequence of short, intermediate and longer-term observable and intended outcomes. This sequencing of outcomes is referred to as an outcome chain. It specifies what needs to change within people or their environments (short to intermediate term impacts) for the longer-term intervention outcome to be achieved. The Change Model explains how and why the change in participants happens. Assess whether the review provides a description of how the interventions work to achieve outcomes with consideration to the activities and strategies that are intended to bring about change. Some reviews will include multiple Change Models.
 Logic diagram usedc Please specify whether the review provides a graphical depiction of how each intervention works to achieve its short, intermediate and long-term outcomes. These diagrams are also referred to as ‘logic models’ or ‘theory of change’ diagrams. The sequence of outcomes (short term to long-term) should be linked to intervention strategies or activities.
Environment (external context)
 Years Years in which primary studies were published. Can be used as a proxy measure for historical/period effects.
 Country Name of county of program delivery. May act as a proxy for political climate, availability of resources, social norms.
 Regions or areas within countries Areas and regions within countries may be specified and may include remoteness or urbanicity indices (e.g., rural, remote/metropolitan, northern/southern). May act as a proxy for access to resources or some other measure.
  1. aStem Question: Are the implementers clearly identified–dichotomous Yes or No response scale
  2. b7 category nominal response scale: 1, 2, 3, 4, 5+, Not specified, Unclear
  3. c4 category nominal response scale: No, Yes (articulation clear), Yes (articulation unclear), Other