Table 2 Process evaluation value created through the socio-technical processes of ‘doing’ the process evaluation

Relationships

Relationships between process evaluation participants

Data collection process building trust and identity within a group of process evaluation participants [25]

Providing process evaluation participants from different research sites opportunities to network with each other [26]

Promoting wider inter-organisational collaboration and learning [27]

*Potential negative consequence of status issues and concerns about repercussions between process evaluation participants in group data collection [25]

Relationships between researchers and process evaluation participants and other stakeholders

Activities such as qualitative interviews and stakeholder involvement in research enhancing trust, communication, and a sense of mutual understanding between researchers and process evaluation participants [25, 26, 28]

Contribute to broader research/practice partnerships and collaborations [29]

*Potential negative consequences of feeding back negative findings to intervention implementers and stakeholders straining relationships and reducing morale and engagement, particularly if not handled sensitively [26, 30]

*Possible tension if stakeholders expect feedback during RCTs but this cannot be provided as it would harm the RCT’s ability to establish causality [30, 31]

*Potential misunderstandings about purpose of evaluation as grading performance rather than learning opportunities [30]

Giving people a voice

Empowerment or disempowerment of process evaluation participants

Asking process evaluation participants how to improve interventions signified they were listened to and empowered, however with the important caveat that their views were acted upon [32]

Promoting the voices of everybody involved, reflecting dignity and validity of multiple viewpoints [25]

Appreciation from process evaluation participants of being asked about their views, experiences, and feelings, about which they were seldom asked [33]

Appreciation from process evaluation participants giving opinions in meetings that clinical leaders also present to hear their voice [34]

*Potential negative consequence of process evaluation participant disempowerment if views not acted on [32], inadequate representation of different stakeholders [35], researcher perspectives privileged [35], researcher use of esoteric language [36], voices perceived as going into a research ‘black hole’ [34]

Education

Educating students

Providing students with opportunities to gain experience in research [37]

Gaining PhDs through conducting process evaluations [38]

Ethical issues

Consent

*Conducting observations in settings where some people are not participants in the evaluation [39]

*Ethical issues around consent for research use of routinely collected clinical data [40]

Confidentiality

*Confidentiality of individual participant responses, and sensitive handling of information that could be detrimental to others [25, 26]

Participant harm

*Potential emotional ill-effects on process evaluation participants such as embarrassment [1], feeling nervous, threatened, uncomfortable being observed [41]

*Disruption and burden to practice settings

Acting on process data suggesting problems with the outcome evaluation

*Ethical dilemmas when process evaluations do not have a formative role but identify problems with an intervention/outcome evaluation [42, 43]

Potential role for process evaluations to monitor the ethical conduct of RCTs [42]

Financial

Inefficiency and waste

*Potential for process evaluations to waste money through inefficiency and collecting too much data [1]

Impact on the outcome evaluation

Increasing likelihood of positive outcome results

Correcting implementation formatively may increase the likelihood of positive outcome results [11, 39, 44, 45]

Realist formative process evaluation in pilot trial resulted in intervention being more adaptable to individual and local contexts and therefore more likely to demonstrate effect in full trial [46]

Realist formative process evaluation in pilot trial providing in-depth implementation and delivery knowledge for main trial [46]

*Potential for certain process evaluation data collection methods such as in-depth interviews to enhance intervention effects [47]

*Potential Hawthorne effects [1]

Increasing staff engagement with the evaluation

Providing feedback to stakeholders through monitoring and quality control may generate enthusiasm, which may be beneficial to the success of the intervention and evaluation [24]

Staff delivering interventions are likely to expect and wish to improve their practice [34, 48], and therefore collaboration to formatively improve interventions may have the value of engaging and motivating staff involvement [34]

Formative process evaluation may help sustain staff interest and engagement in trials lasting several years [49]

Formative improvement of trial processes likely to enhance cooperation of staff collecting process data and timely correction of problems which threaten the evaluation [50]

Adding burden to outcome evaluation staff and participants

*Potential burden of process evaluation data collection on intervention staff and participants [1, 7]

Meeting a requirement

Fulfilling a requirement to include a process evaluation from funding bodies and research commissioners [51, 52], guidance [2, 53], or calls within fields [54, 55]

Adding bias to outcome evaluation

*Outcome evaluators gaining insight into how the intervention is functioning which may bias their interpretation of outcomes [1, 56]

*Possibility of unblinded process evaluators accidentally revealing participant allocations to outcome evaluators [1]