Skip to main content

Developing strategies to improve fidelity of delivery of, and engagement with, a complex intervention to improve independence in dementia: a mixed methods study

Abstract

Background

It is important to evaluate fidelity of delivery and engagement during feasibility trials. However, there is little guidance on how to systematically develop strategies to improve implementation if problems arise. We aimed to: 1) Assess fidelity of delivery and engagement, 2) Identify factors influencing fidelity of delivery and engagement, and 3) Develop strategies to improve fidelity of delivery of, and engagement with, a complex intervention to improve independence in dementia, within a feasibility trial.

Methods

A mixed methods evaluation of an intervention that aimed to improve independence in dementia. To assess fidelity of delivery and engagement, observation and self-report methods were used: 60% of audio-recorded intervention sessions were transcribed and reliably rated for fidelity. Providers (n = 12) and people with dementia/supporters (n = 34) were asked to complete checklists after each session. Descriptive statistics were used to analyse the data. To identify factors influencing fidelity and engagement, one-to-one semi-structured interviews were conducted with providers (n = 8), people with dementia (n = 7) and supporters (n = 7). Thematic analysis and content analysis were used to analyse data. To develop strategies, we followed four steps proposed by the authors of the Behaviour Change Wheel (1. Understanding the behaviour, 2. Identifying intervention functions, 3. Specifying intervention content, 4. Identifying mode of delivery).

Results

Researcher ratings indicated moderate fidelity and provider/participant ratings indicated high fidelity of delivery. Knowledge, providers’ attributes, ease of adaptation of the intervention in relation to participants’ needs and logistical considerations influenced fidelity. We developed four strategies to improve fidelity of delivery of PRIDE: 1) showing a video, 2) giving an instruction sheet, 3) giving time to practice and 4) providing continued support. Participants reported high levels of engagement. Participants’ attributes, capability and opportunity influenced engagement. We developed four strategies to improve engagement with PRIDE: 1) a session summary document, 2) clear instructions, 3) time to practice activity and 4) providing regular compulsory telephone support.

Conclusion

Fidelity of delivery and engagement are complex behaviours. This manuscript provides an example of how the Behaviour Change Wheel can be used during a feasibility trial to systematically develop strategies to improve implementation of complex interventions.

Peer Review reports

Contributions to the literature

  • These findings highlight the complexity of fidelity of delivery and engagement behaviours in complex health interventions.

  • This is one of few examples which have used fidelity data to identify issues and develop evidence-based strategies to improve intervention delivery and engagement during a feasibility trial.

  • This study provides an example of how mixed methods evaluations can be used to develop strategies to improve implementation for a complex dementia intervention.

Background

Complex health interventions have the potential to improve outcomes for intervention participants. These complex health interventions are often adapted, e.g. to facilitate the implementation of interventions in a different country [1]. Adaptations may also be considered prior to evaluating an intervention. For example: MRC guidance on developing and evaluating complex interventions states that problems with an intervention (e.g. acceptability, compliance and delivery) can be identified during the feasibility trial and refined or adapted before being delivered and evaluated in an RCT [2]. However, there is little guidance on how to adapt interventions systematically to refine interventions based on problems identified during a feasibility trial.

Findings from evaluations of complex health interventions indicate that interventions are often not delivered as planned (termed ‘fidelity of delivery’) [3,4,5]. The term engagement is used in this paper as an umbrella term [6] to refer to whether participants understand the information (termed ‘receipt’) and can put their plans into practice (termed ‘enactment’) [7]. Fidelity of delivery refers to the intervention providers’ performance and engagement relates to the participants’ performance. To maximise the potential for effectiveness, it is necessary to understand whether interventions are delivered as planned and engaged with, and develop ways to improve fidelity and engagement if implementation problems arise.

Intervention ‘fidelity’ (including fidelity of delivery and engagement) is one element of process evaluations [8]. Process evaluation is recommended at all stage of intervention evaluation, including feasibility, effectiveness and implementation. Fidelity of delivery and engagement are complex behaviours with many underlying mechanisms. For example, environmental, organisational and individual characteristics have been found to influence fidelity of, and engagement with dementia interventions and interventions in general [9,10,11,12,13,14,15,16,17,18]. To understand how an intervention is implemented, the Medical Research Council (MRC) guidance [8], together with previous research [15, 19] propose that mixed methods evaluations of interventions are needed to provide a detailed understanding of intervention functions. Quantitative methods such as observation and self-report can be used to find out whether interventions are delivered as planned (fidelity of delivery) or engaged with [6]; with one method of fidelity of delivery being to audio-record sessions and transcribe and rate a random proportion against a pre-defined intervention checklist [4, 7]. To overcome limitations of individual methods, the use of multiple methods are recommended [20,21,22,23]. Alternatively, qualitative methods can be used to gain an in-depth understanding of factors influencing fidelity, including reasons for differences in fidelity or engagement across participants, providers and contexts [8, 23]. By integrating data from both quantitative and qualitative studies, strategies to improve fidelity of delivery of, and engagement with, a feasibility trial could be developed.

Behaviours take place within complex systems [24], therefore changing behaviour is difficult [25]. To develop evidence-based effective strategies for changing behaviour, theory needs to be drawn upon. To take complexities of fidelity and engagement into account, theory needs to include intrapersonal, interpersonal and environmental factors. Individual behaviour change theories are not suitable for this purpose. Instead, a framework of behaviour change can be applied. One such integrated framework is the Behaviour Change Wheel (BCW) [26]. The BCW has three levels: The COM-B (Capability, Opportunity, Motivation – Behaviour) model, nine intervention functions and seven policy categories. The BCW provides a systematic approach to intervention development that can be applied to different populations and behaviours and has previously been used to change behaviours related to fidelity (e.g. [27]) and engagement (e.g. [28,29,30]). However, to the authors’ knowledge, no research has followed the stages proposed by the authors of the BCW to develop strategies to improve fidelity of delivery of, and engagement with, a feasibility trial.

Previous research has used mixed methods to explore fidelity of delivery of, and engagement with complex health interventions (e.g. [15, 31, 32]). However, to the authors’ knowledge, this is the first study to integrate findings to develop behavioural strategies to improve fidelity of delivery of, and engagement with, a complex dementia intervention. This study extends previous research by using a theory-based, systematic method to develop preliminary strategies which could be used to improve fidelity of delivery of, and engagement with, a future PRIDE RCT.

The strategies outlined in this manuscript are to be considered within the context of improving the Promoting Independence in Dementia (PRIDE) intervention [33, 34]. The PRIDE intervention encourages people living with dementia and supporters (family member/spouse/friend) to identify idiosyncratic goals to increase independence. The PRIDE feasibility trial tested a three session intervention delivered by dementia advice workers (staff working in or alongside memory clinics; here referred to as ‘providers’) to people with dementia and their supporter [33]. The intervention is a manualised, tailored intervention. Dyads (the person living with dementia and their supporter) chose three topics to work on. Topic options included: 1) Keeping mentally active, 2) Keeping physically active, 3) Keeping socially active, 4) Making decisions, 5) Getting the message across, 6) Receiving a diagnosis of dementia and 7) Keeping healthy. In the first session, providers provided information and encouraged participants to choose three topics and plan one or more activities to work on. In the following sessions, providers and participants reviewed plans and engaged in problem solving before developing new plans [34]. The intervention contained both standardised components that were delivered to all participants and tailored components that were dependent on which topics participants chose. Details of intervention components are reported in the PRIDE fidelity checklist development paper [35].

Using PRIDE as a model, this study aimed to demonstrate one way in which strategies can be developed to improve fidelity of delivery of, and engagement with, a feasibility trial. In this paper we use the PRIDE fidelity assessment as an example. The PRIDE fidelity assessment aimed to:

  1. 1.

    Assess fidelity of delivery of, and engagement with the PRIDE intervention

  2. 2.

    Identify factors influencing fidelity of delivery of, and engagement with the PRIDE intervention

  3. 3.

    Develop strategies to improve fidelity of delivery of, and engagement with the PRIDE intervention.

Methods

Ethics

Ethical and research governance requirements were followed. Data were transcribed professionally and all transcripts were fully anonymised. Individuals were unidentifiable from data or resulting outputs. Ethical approval was obtained from the NHS East Midlands – Nottingham 1 Research Ethics committee (REC reference number: 16/EM/0044). Data were accessed by authorised study members and stored securely in a central location.

Design

An explanatory sequential mixed methods design was used [23]. Firstly, we conducted a longitudinal observational study assessing how providers delivered the intervention and how participants engaged with the intervention. Then, we conducted a qualitative semi-structured interview study to understand barriers and facilitators to fidelity of delivery and engagement. The findings from both of these studies were then integrated using a behaviour change framework with the purpose of identifying and understanding potential behavioural strategies to improve fidelity of delivery and engagement (see Fig. 1). This is consistent with Creswell’s [23] definition of mixed methods research which involves the collection and integration of quantitative and qualitative data to draw interpretations from the strengths of both datasets to understand research problems.

Fig. 1
figure1

Flow chart outlining how the different parts of the studies link together and how findings from the first two stages informed the third stage

1) Assessing fidelity of delivery of, and engagement with PRIDE

Further details on this stage of research can be found in the first author’s PhD thesis [36].

Measures

To measure fidelity of delivery and engagement, reliable checklists (containing all intervention components) and coding guidelines that can be used by researchers, providers and participants were developed specifically for this study [35].

Sample

Sixty percent of all PRIDE intervention sessions: 24 sets (six sets of three sessions from each of the four sites) were randomly sampled. Previous research recommends that researchers monitor fidelity of delivery for 20–40% of intervention sessions [37]. To identify whether fidelity of delivery and engagement varied across participants, providers and sites, data were collected from all four sites. All providers (n = 12) and people with dementia/supporters (n = 34) were asked to complete fidelity checklists after each session (see [35] for response rates).

Procedure

All intervention sessions were recorded (with written consent from people with dementia and supporters). Providers stopped recording if it was inappropriate during the sessions (e.g. participant distress). Audio-recordings were stored securely and transcribed by a professional company. Transcripts were checked for accuracy and fully anonymised (by HW).

HW coded all transcripts to determine whether intervention components were delivered. To identify coder drift (changes in coder agreement over time), a second researcher (MW) double-coded 10% of sets that were not double-coded during measure development (see 35 for details of transcripts double-coded during measure development). Inter-rater agreement was calculated using weighted kappa and percentage agreement. If raters did not achieve weighted kappa scores ≥0.61, further sets were coded and discussed until agreement was reached [35].

Providers were trained to complete fidelity checklists (by HW). This training took place at the end of the PRIDE training day. All providers received the same training for fidelity assessments. Training included details on why it is important to measure fidelity and how to audio-record and complete checklists. Provider and participant checklists were returned by email, via a secure electronic system (providers only) or post (using prepaid envelope).

Analysis method

Details on the development of coding guidelines and scores are reported in the PRIDE fidelity checklist development paper (see [35]).

Descriptive statistics were used to assess the percentage of standardised components that were delivered as planned per session (fidelity of delivery). These were compared across sessions (Session 1, Session 2, Session 3), providers and sites. Fidelity of delivery was assessed using data from researchers, providers and the person with dementia (resulting in three fidelity scores for each session). We calculated a percentage and total score for the number of standardised components delivered (80–100% high fidelity, 51–80% moderate fidelity, < 50% low fidelity) [7]. We rated components according to whether they were ‘done’/‘definitely happened’ (Score 2), ‘done to some extent’/‘possibly happened’ (Score 1), ‘not done’/‘didn’t happen’ (Score 0), ‘not applicable’ (Coded 97), ‘missing (Coded -999), or ‘unclear’ (Coded 10). We measured the delivery of tailored topics and components (for Session 1 and Session 2) by calculating the average number of tailored topics delivered in a session and number of tailored components delivered for each topic.

Descriptive statistics were used to assess levels of engagement. We measured intervention receipt (‘information was clear and easy to understand’/‘understanding how to put plans into practice’) and intervention enactment (‘recorded activities on calendar’/‘practiced and used information and skills learnt’). Engagement was assessed using data from the person with dementia’s self-reported checklists. We compared percentages of those who chose ‘yes’ (Score 2), ‘to some extent’ (Score 1), ‘no’ (Score 0), ‘not applicable’ (Coded 97), ‘missing’ (Coded − 999) or ‘unclear’ (Coded 10) across providers and sites.

Missing data, unclear data and components that were not applicable were scored as ‘0’ (not done). This was to ensure that conservative estimates of fidelity of delivery and engagement were provided and to ensure that findings was comparable across providers and sites.

2) Identifying barriers and facilitators to fidelity of delivery of and engagement with PRIDE

Further details on this stage of research can be found in the first author’s PhD thesis [36].

Sample

We used purposive opportunity sampling to ensure that providers and people with dementia-supporter dyads were recruited from all four sites. PRIDE providers and participants were invited to take part.

In total, eight providers, seven people with dementia and seven supporters were recruited (see Table 1).

Table 1 Demographic characteristics of providers, people with dementia and supporters for one-to-one interviews

When potential participants expressed an interest, both members of the dyad were asked if they would like to take part. We interviewed people with dementia and their supporters separately where possible. Both the participant and supporter were present in seven interviews (three person with dementia interviews and four supporter interviews).

Measures

Two semi-structured interview guides were developed. The interview schedule for providers was developed to explore barriers and facilitators to the delivery of PRIDE, and experiences of delivery (see Supplementary materials 1). The interview schedule for people with dementia and supporters explored experiences of participating, and barriers and facilitators to engaging with PRIDE (see Supplementary materials 2). Questions mapped onto COM-B [26] and Theoretical Domain Framework (TDF) domains [38]. The PRIDE team and co-authors (AS/IT/SM) provided feedback on questions. We iteratively revised the interview schedules throughout data collection.

Procedure

Providers were invited to participate after their last dyads’ final intervention session. People with dementia and their supporters were invited to participate during the follow-up session. Potential participants were contacted by telephone to provide more details and arrange a date and convenient location for the interviews (e.g. participants’ homes or workplace). To ensure that participants were still familiar with the intervention, we aimed to conduct interviews 1–2 weeks after the dyads’ final session, or as soon as possible after the provider delivered their last session. In practice, this was not possible due to timing of follow-up visits. Therefore, interviews took place as soon as possible after the dyads received information about the interview study.

Participants gave written informed consent (including consent for audio-recording) at the beginning of the interview. All participants were in the mild stages of dementia and could provide consent. Supporters also provided secondary consent for the person with dementia, as requested by the ethics committee. We did not collect demographic characteristics unless they were volunteered during the interviews. Participants’ age and providers’ amount of experience were prompted and recorded [39].

Twenty-two interviews were conducted by HW (12–93 min). All interviews were audio-recorded. To prompt discussion, the dyads’ copy of the PRIDE intervention manuals, intervention sheets and fidelity checklists were used. For example, the PRIDE manual goal-setting page was sometimes used to prompt discussion around which activities were chosen. Audio-recordings were transcribed verbatim by a professional company. Transcripts were checked for accuracy and fully anonymised (names and places). Codes were assigned to each participant (e.g. P1: Person with dementia 1).

Analysis method

To analyse interview data, thematic analysis and content analysis were used.

Thematic analysis

This study combined inductive thematic analysis (as proposed by Braun and Clarke) [40] with elements of deductive thematic analysis (e.g. the use of a coding frame) [41]. This approach would fit within a ‘medium Q thematic analysis approach’. [42] The methods of thematic analysis were facilitated by the iterative categorisation method; a systematic, rigorous and transparent technique for sorting data [43].

Transcripts were read and re-read to become familiar with the data. Line-by-line coding was then conducted for 12 interviews (four provider interviews, four participant interviews and four supporter interviews) to generate initial codes. We then developed an initial coding framework from this inductive coding (one for delivery, one for engagement) [41]. Two researchers (HW/JB) independently applied the coding framework to three transcripts (one provider, one person with dementia, one supporter). Differences were discussed and resolved. Minor changes were made to the coding framework. Then, HW coded all transcripts (n = 22) using the final coding framework using NVivo 11. From this coding, themes were developed, reviewed, defined and labelled. A table of themes, sub-themes and example quotes was created for both fidelity of delivery and engagement. Themes and example quotes were reviewed by JB and discussed with the wider research team.

Previous research has suggested that reliability assessments may not be appropriate [40, 44] as they may restrict coding flexibility and the identification of new findings [45]. Instead, to enhance trustworthiness of data analysis and analysis interpretation, a second researcher (JB) was involved in applying the coding frame and peer reviewing coding and summaries [46,47,48,49].

Content analysis

Deductive content analysis (categorisation of data to pre-defined categories) [50] was also used to identify which COM-B domains were frequently reported as barriers/facilitators to delivery and engagement. Data was categorised into: capability (psychological or physical), opportunity (physical or social) and motivation (automatic or reflective) [26]. All of the raw data quotes were extracted into a spreadsheet and duplicates were removed. Two authors (HW/AR) coded all extracts into COM-B categories and as barriers/facilitators. Discrepancies were resolved through discussion. Quotes that did not fit into any domains were coded as ‘none’. More than one domain could be coded for each quote. Frequency of each category was identified.

3) Developing strategies to improve fidelity of delivery and engagement

To improve fidelity of delivery of, and engagement with, PRIDE, we developed initial strategies using the steps proposed by the authors of the BCW [25]. These strategies were designed to inform improvements to a future PRIDE RCT. Steps include: 1) Understanding the behaviour, 2) Identifying intervention functions and policy categories, 3) Specifying intervention content in terms of Behaviour Change Techniques (BCTs: active intervention ingredients) and 4) Identifying a mode of delivery [25].

Firstly, the target behaviours were defined and barriers and facilitators for these behaviours were identified (Step 1). To identify appropriate intervention functions, the BCW was used to highlight optimal links between barriers/facilitators and intervention functions (Step 2). To identify appropriate behaviour change techniques, the BCW was used to highlight optimal links between intervention functions and behaviour change techniques (Step 3). Suitable modes of delivery for individual strategies were then identified (Step 4) and strategies to improve fidelity of delivery and engagement were developed (Step 5) [25]. Findings from the quantitative fidelity/engagement assessment (stage 1) and qualitative findings (stage 2), together with previous research in similar populations were used to identify and narrow down appropriate solutions by judging the suitability of different options (in Steps 2–4). Each of these steps (including how findings from both the quantitative and qualitative studies were integrated) are described in more detail in Fig. 1. Further details on this stage of research can be found in the first author’s PhD thesis [36].

Results

1) Assessing fidelity of delivery of, and engagement with PRIDE

Fidelity of delivery

Standardised components

Ratings from transcripts of audio-recordings indicated that a mean of 69% of components were delivered in Session 1 (range: 13.6–86.4%), 57.7% in Session 2 (range: 41.7–83.3%) and 54.9% in Session 3 (range: 25–95.8%). Provider self-report ratings indicated that a mean of 85% of components were delivered in Session 1 (range: 22.7–100%), 84.3% in Session 2 (range: 61.1–100%) and 86.5% in Session 3 (range: 62.5–100%). Participant self-report ratings indicated that a mean of 89.8% of components were delivered in Session 1 (range: 59.1–100%), 90.1% in Session 2 (range: 50–100%), and 92.5% in Session 3 (range: 50–100%). Findings indicate that providers delivered PRIDE with moderate to high fidelity; with researcher ratings indicating the lowest levels and participant self-report ratings indicating the highest levels of fidelity. Mean fidelity for each session varied across sites and providers across all three sources of rating (see Supplementary materials 3 or [36] for further details).

Tailored components

Ratings from transcripts of audio-recordings indicated that a mean of 1.9 topics and 4.6 components were delivered in Session 1 and 1.1 topics and 3.0 components in Session 2. Provider self-report ratings indicated that a mean of 2.4 topics and 7.9 components were delivered in Session 1 and 2.2 topics and 7.8 components in Session 2. The number of tailored topics and components delivered varied across sites and providers (see Supplementary materials 4 or [36] for further details).

Engagement

Findings from the self-report checklists completed by participants with dementia indicated that the mean level of receipt was 85.9% for Session 1 (range: 0–100%), 87.5% for Session 2 (range: 50–100%) and 90.6% for Session 3 (range: 50–100%). The mean level of enactment was 81.3% for Session 2 (range: 0–100%) and 82.8% for Session 3 (range: 0–100%). Findings indicate that participants understood information and were able to put plans into practice (see Supplementary materials 5 or [36] for further details).

2) Identifying barriers and facilitators to fidelity of delivery of and engagement with PRIDE

Fidelity of delivery

Thematic analysis

Four themes were developed. Examples quotes are shown in Table 2. More details from this analysis can be found in the first author’s PhD thesis [36].

  1. I.

    Providers’ knowledge about how to deliver PRIDE

Table 2 Themes, sub-themes and example quotes for fidelity of delivery

Providers’ previous qualifications and/or experience of working with people with dementia helped them to deliver PRIDE with fidelity by increasing confidence and ability to tailor the intervention. Prior knowledge of individual participants and the local environment helped them to deliver PRIDE as planned (e.g. knowledge of activities in local area).

Whilst providers understood how to complete forms, a lack of knowledge limited their skills to deliver PRIDE or use the manual; thus making it more difficult to deliver the intervention with fidelity. Providers expressed a need for further training on the practical elements of delivery (e.g. delivering key information and adapting this to participants, or delivering PRIDE using the manual). Practice and familiarisation with the manual and practice developed confidence in delivering PRIDE, with delivery becoming easier over time.

  1. II.

    Providers’ personal attributes

Providers’ perceptions that PRIDE was different to their job role made it more difficult for providers to deliver PRIDE as planned (e.g. PRIDE not being in their remit and not being used to following a manual), whereas similarities made it easier to deliver (e.g. delivering similar content or dealing with similar issues that exist within providers’ current job roles).

Personal characteristics (e.g. feeling conscious about delivering the intervention as planned, wanting to stay longer with participants), and personal views on benefits for themselves (e.g. wanting to learn more, enjoying spending time with people with dementia) and the participants (e.g. having potential to change post-diagnostic support) influenced delivery.

Some providers were anxious about delivering PRIDE for many reasons including being judged by participants and being recorded/filling in fidelity checklists. On the other hand, perceived self-confidence facilitated delivery; including age, work-related experience and experience delivering PRIDE.

  1. III.

    Adaptation of PRIDE in relation to participants’ needs

There was conflict between trying to deliver PRIDE as planned and delivering PRIDE in the most appropriate way for participants. As it was difficult to deliver PRIDE as planned, providers delivered the intervention flexibly, with participants’ needs in mind. Delivering PRIDE with strict fidelity felt restricting and providers were more comfortable delivering PRIDE using their own style. Having experience helped providers to deliver PRIDE and facilitated the adaptation of PRIDE in relation to participants’ needs, but may also have hindered the delivery of PRIDE as planned in the manual. Providers initially delivered the intervention with strict fidelity but became less reliant on the manual and delivered PRIDE flexibly over time.

Participants’ understanding, motivation and engagement influenced fidelity of delivery. There was a lack of consensus among providers regarding the appropriateness of PRIDE for people with different stages of dementia. PRIDE was perceived as suitable to those with a diagnosis of mild dementia if the characteristics of dementia (e.g. level of cognitive impairment) were considered when delivering the intervention. PRIDE was seen as more difficult to deliver in some situations (e.g. supporters not being present in sessions).

  1. IV.

    Logistical considerations

PRIDE mostly fitted well around providers’ work commitments. Facilitators of fidelity included having a supportive work environment, managing their own diaries, and the allocation of appropriate time for each dyad. Lack of time to prepare for sessions and travel to dyads, and difficulties scheduling three sessions at the right time in the month hindered fidelity of delivery.

The PRIDE research team, site researchers and other providers facilitated fidelity of delivery. Peer support from other providers offered an opportunity to share experiences, gain knowledge about delivering PRIDE and reduce anxieties. More support with technical difficulties from researchers or peers may have been beneficial (e.g. support with Dictaphones). There was no consensus regarding what role supporters should have; with findings indicating that the presence of a supporter should depend on individual needs. Providers felt that supporters facilitated delivery (e.g. providing practical support for engaging in activities). Others reported that sometimes supporters were involved too much (e.g. telling own story in session which reduced intervention time) or too little (e.g. not being there during sessions made it hard to make progress).

Content analysis

For fidelity, frequent barriers related to social opportunity (n = 131), physical opportunity (n = 123) and psychological capability (n = 85). Frequent facilitators related to social opportunity (n = 162), physical opportunity (n = 110), reflective motivation (n = 107) and psychological capability. The frequency of occurrences of COM-B domains are reported in Table 3 (see Supplementary materials 6/7).

Table 3 Frequency of occurrences of COM-B domains in total and by barriers to and facilitators for fidelity of delivery

Engagement

Thematic analysis

Three themes were developed. Examples quotes are shown in Table 4. More details from this analysis can be found in the first author’s PhD thesis [36].

  1. I.

    Participants’ attributes

Table 4 Themes, sub-themes and example quotes for engagement

People with dementia and supporters reported taking part in activities that they wanted or like to do (e.g. those that they have experience with, those that are convenient and easy, or social activities) and avoided activities they disliked. However, if participants were unable to perform the activities, liking activities was not sufficient to influence engagement. Personality traits (including competitiveness, sportiness, sociability and thoroughness) influenced decisions to engage in activities.

People with dementia and supporters’ beliefs about PRIDE facilitated engagement. They felt that PRIDE and its components were helpful in encouraging them to do activities, having someone to talk to and for helping others. If participants felt that PRIDE was relevant to them, this facilitated engagement. Despite the manual facilitating engagement, participants reported that some aspects of the manual were not relevant for them. Some participants felt that the manual/intervention content would become more relevant as symptoms progress.

Enjoyment of PRIDE sessions and spending time with the provider, and enjoyment of taking part in their chosen activities helped participants to engage. However, anxiety towards activities chosen during PRIDE sessions was perceived to limit engagement activities for people with dementia and their supporters. Conversely, anxieties (e.g. about doing activities, travelling to activities and using technology) and negative feelings (in relation to memory impairment, inability to go out and do things and not enjoying company) limited engagement.

  1. II.

    Participants’ capability

Physical health problems challenged engagement with certain activities whereas good health facilitated engagement. Cognitive factors including memory (e.g. not remembering sessions and intervention materials, or memory affecting ability to engage with activities), knowledge (e.g. understanding information in sessions and not knowing how to get to activities, or do activities) and communication skills were perceived to influence engagement. Familiarity with activities was perceived to facilitate engagement. Participants discussed engaging in activities that they do not do as frequently as they used to, but that they still have the capability to do.

  1. III.

    Participants’ opportunity to engage

Accessibility, including accessibility of PRIDE materials and activities influenced engagement. Participants felt that PRIDE paperwork was easy to complete and the manual was easy to understand; thus facilitating engagement. The location and accessibility of activities helped or hindered people with dementia and supporters from engaging with activities (e.g. it helped if participants could travel to the activity easily). Having the physical resources needed to engage with activities helped people with dementia and their supporters to do their activities (e.g. using reminders, writing notes, or having the necessary tools). However, cost, time and challenges finding the appropriate equipment hindered engagement.

Practical support from the provider (e.g. encouraging them to do their activity, or giving information) helped participants to engage with the intervention sessions and activities. Practical support from supporters (e.g. setting up activities, researching and organising activities, helping them get to the activity and help to complete forms) also facilitated engagement for people with dementia. Practical support from family members included accompanying the person with dementia to their activity to overcome travel barriers. Emotional support from providers, family members and organisations facilitated engagement. For example, having a positive relationship with their provider facilitated engagement by giving participants someone to talk to during PRIDE. In some cases, further support was perceived to be necessary (e.g. having someone to do the activity with).

Content analysis

For engagement, frequent barriers related to physical opportunity (n = 102) and psychological capability (n = 100). Frequent facilitators related to social opportunity (n = 188), reflective motivation (n = 144), physical opportunity (n = 111) and automatic motivation (n = 93). The frequency of occurrences of COM-B domains are reported in Table 5 (see Supplementary materials 8/9).

Table 5 Frequency of occurrences of COM-B domains in total and by barriers to, and facilitators for engagement

3) Developing strategies to improve fidelity of delivery and engagement with PRIDE

Fidelity of delivery

Findings from the fidelity assessment and interviews were integrated to develop improvement strategies. Further details can be found in the first author’s PhD thesis [36].

The target behaviour that strategies aimed to improve was that providers would deliver PRIDE components that were infrequently delivered within the PRIDE feasibility trial. These components related to tailoring the intervention to participants (providing resources for chosen topics and discussing these in relation to the participants) and problem solving. Three intervention functions were identified as potentially relevant to include within recommendations to improve key barriers, including: psychological capability (e.g. skills to deliver PRIDE as planned and remembering information from training), social opportunity (e.g. participant engagement and support from researchers) and physical opportunity (e.g. having the appropriate PRIDE resources and time to practice delivering PRIDE) to deliver PRIDE as planned. These were ‘Training’, ‘Modelling’ and ‘Enablement’. One policy category was identified as relevant: ‘Service provision’. Four BCTs from the BCTTv1 [51] were identified to include within recommendations. These were: ‘Social support (unspecified)’ (BCT 3.1), ‘Instruction on how to perform the behaviour’ (BCT 4.1), ‘Demonstration of behaviour’ (BCT 6.1), and ‘Behavioural practice and rehearsal’ (BCT 8.1). Four different types of mode were identified as suitable to deliver the four BCTs. These were: human interactions (e.g. face-to-face or over the telephone), printed materials and digital delivery (see Table 6 for details of each step and rationale).

Table 6 Mapping of fidelity findings and previous research onto Steps 1–4 of the BCW [25], along with resulting recommendations

In total, four strategies were developed to improve fidelity of delivery. These were: 1) show a video of how to deliver PRIDE, 2) give an instruction sheet about how to deliver PRIDE, 3) give providers time to practice delivering PRIDE within the training session, and 4) provide continued support from researchers for delivery. Table 6 (Step 5) provides more details of recommendations and potential implementation.

Engagement

The target behaviour was that people with dementia and supporters would carry out the activities that they planned in the first or second PRIDE sessions (e.g. attending an activity group or completing a jigsaw puzzle). Three intervention functions were identified as potentially relevant to include within strategies to improve key barriers, including: physical opportunity (money and time to do the activity, accessible locations and resources to prompt activities) and psychological capability (knowledge and skills about how to do activities and how to organise and carry out activities). These were ‘Education’, ‘Training’ and ‘Enablement’. One policy category was identified as relevant: ‘Service provision’. Five BCTs were identified to include within recommendations. These were: ‘Social support (unspecified)’ (BCT 3.1), ‘Social support (practical)’ (BCT 3.2), ‘Instruction on how to perform the behaviour’ (BCT 4.1), ‘Prompts and cues’ (BCT 7.1), and ‘Behavioural practice and rehearsal’ (BCT 8.1). Three different types of mode of delivery were identified as suitable to deliver the five BCTs. These were: face-to-face human interactions, human interactions remotely via telephone and printed materials (see Table 7 for details of each step and rationale).

Table 7 Mapping of engagement findings and previous research to steps 1–4 of BCW [25], and resulting recommendations

In total, four strategies were developed to improve engagement: 1) give participants a session summary document, 2) give participants clear instructions detailing how to do their chosen activity, 3) ensure that there is time within the PRIDE sessions to practice the chosen activity where possible, and 4) provide regular compulsory telephone support from the provider. Table 7 (Step 5) provides more details of recommendations and potential implementation.

Discussion

Key findings

This manuscript aimed to outline a mixed-methods process that can be used to systematically develop recommendations to improve fidelity of delivery and engagement. This manuscript outlined fidelity of delivery and engagement findings from an intervention which aimed to improve independence in dementia [33, 34]. Researcher ratings indicated moderate fidelity of delivery and provider and participant rating indicated high fidelity of delivery. Several factors influenced fidelity of delivery including knowledge, personal attributes, skills and a supportive work environment. Engagement among people with dementia and supporters was high. Many factors influenced engagement including personal attributes, capability, and opportunity to engage with PRIDE and related activities. Four strategies to improve fidelity and four strategies to improve engagement were developed using the BCW [25]. Whilst these strategies were developed specifically for PRIDE, the methods used to develop strategies can be applied to other complex interventions (see [36] for further details).

How findings relate to previous research

This study extends previous knowledge [2] by using a theory-based, systematic method to develop preliminary strategies that could be used to improve fidelity of delivery of, and engagement with, a future PRIDE randomised control trial, as well as other evaluations of complex interventions. The resulting outcomes provide detailed and transparent information about (i) strategies that were developed, and (ii) the associated BCTs and intervention functions [69]. This information could inform PRIDE researchers’ decisions when considering intervention refinement. In addition, the methods outlined in this manuscript can be used more widely by other intervention developers and evaluators to identify problems with implementation during the feasibility stage and to develop strategies to potentially improve fidelity and engagement.

Whilst strategies developed using the Behaviour Change Wheel largely focus on overcoming individual barriers for providers and participants (e.g. knowledge), some strategies also target organisational factors (e.g. providers (not having time within the work role to practice delivering PRIDE as planned). Targeting organisational barriers is important given that these findings, along with previous research findings demonstrate organisational barriers to fidelity of delivery [12, 17, 55] and environmental barriers to engagement [11, 18, 70, 71]. Additionally, the level of detail and adaptation with which strategies need to be implemented will differ for individual participants and providers depending on their individual needs.

Our findings demonstrate the complexity of fidelity of delivery and engagement behaviours. There were many factors found to influence fidelity of, and engagement with, PRIDE; despite PRIDE being moderately delivered as planned and engaged with by participants. Our finding that PRIDE was delivered with at least moderate fidelity supported previous research, which indicates that interventions are often not delivered completely as planned [3,4,5, 10]. Similarly, our findings of self-reported high levels of engagement refute findings that suggest that older adults find it difficult to engage with information given by healthcare professionals in health appointments [72,73,74]. Our qualitative findings provided support for research which has suggested that a range of factors including intervention factors, provider factors, participant factors and organisational factors influence both fidelity and engagement [13, 14, 17].

Strengths and limitations

A key strength of this research is that it used mixed methods. By conducting a thorough interview study alongside the measurement of fidelity and engagement, these findings provided better insights into what was delivered and engaged with and why.

One limitation of the framework used in this research is that links between intervention functions and BCTs are based on expert consensus [25], and some of the identified BCTs may not be effective in particular populations, behaviours or settings. However, as demonstrated, there is some evidence from previous research which took place in different populations, settings and/or behaviours which suggests that the selected intervention functions and BCTs may be effective for improving fidelity of, and engagement with, PRIDE.

Implications

Wider implications

The BCW provided a systematic approach for developing strategies. This behavioural science approach may be appropriate for use in other process evaluations of complex interventions. The mixed-methods process outlined in this manuscript can be applied by other intervention developers and evaluators to identify problems with implementation during the feasibility stage and to develop strategies to potentially improve fidelity and engagement.

Implications for PRIDE

The strategies outlined in this manuscript, together with the findings from this study can be used to improve training for providers of PRIDE. This would ensure that healthcare providers have the required skills, attributes and facilitative work environment to deliver and adapt PRIDE as planned.

For people with dementia and their supporters, the findings highlight difficulties people face when trying to engage in activities with PRIDE. These factors may also be relevant for people with dementia and supporters who did not take part in PRIDE, but who are trying to find ways to engage in activities. This knowledge can be used to ensure that interventions and services are developed and delivered in a way that reduces barriers and maximises people’s potential to engage.

Future research

This method could be replicated in other interventions to develop strategies to improve fidelity of delivery and engagement. Additionally, researchers could test whether the implementation of such strategies might improve fidelity of delivery and engagement. For example, the initial strategies to improve fidelity of delivery and engagement developed in this mixed methods study, could be iteratively revised following stakeholder feedback, implemented and evaluated in future versions of PRIDE.

Conclusions

Fidelity of delivery and engagement are complex behaviours. This manuscript provides an example of how behavioural science principles and mixed-methods can be applied to complex interventions during the feasibility stage to measure, understand and potentially improve fidelity of delivery and engagement behaviours, thus potentially increasing the effectiveness of interventions and quality of life for participants.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due to participant confidentiality but are available from the corresponding author on reasonable request.

Abbreviations

BCW:

Behaviour Change Wheel

COM-B model:

Capability, Opportunity, Motivation – Behaviour model

MRC:

Medical Research Council

PRIDE:

Promoting Independence in Dementia

TDF:

Theoretical Domains Framework

BCTs:

Behaviour Change Techniques

BCTTv1:

Behaviour Change Technique Taxonomy version 1

References

  1. 1.

    Wenborn J, Hynes S, Moniz-Cook E, Mountain G, Poland F, King M, Omar R, Morris S, Vernooij-Dassen M, Challis D, Michie S. Community occupational therapy for people with dementia and family carers (COTiD-UK) versus treatment as usual (valuing active life in dementia [VALID] programme): study protocol for a randomised controlled trial. Trials. 2016;17(1):65 https://doi.org/10.1186/s13063-015-1150-y.

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  2. 2.

    Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655. https://doi.org/10.1136/bmj.a1655.

    PubMed  PubMed Central  Article  Google Scholar 

  3. 3.

    Lorencatto F, West R, Christopherson C, Michie S. Assessing fidelity of delivery of smoking cessation behavioural support in practice. Implement Sci. 2013;8(40). https://doi.org/10.1186/1748-5908-8-40.

  4. 4.

    Lorencatto F, West R, Bruguera C, Michie S. A method for assessing fidelity of delivery of telephone behavioral support for smoking cessation. J Consult Clin Psychol. 2014;82(3):482. https://doi.org/10.1037/a0035149.

    Article  PubMed  Google Scholar 

  5. 5.

    Toomey E, Currie-Murphy L, Matthews J, Hurley DA. Implementation fidelity of physiotherapist-delivered group education and exercise interventions to promote self-management in people with osteoarthritis and chronic low back pain: a rapid review part II. Man Ther. 2015;20(2):287–94 https://doi.org/10.1016/j.math.2014.10.012.

    PubMed  Article  Google Scholar 

  6. 6.

    Walton H, Spector A, Tombor I, Michie S. Measures of fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions: a systematic review of measure quality. Br J Health Psychol. 2017;22(4):872–903. https://doi.org/10.1111/bjhp.12260.

    Article  PubMed  PubMed Central  Google Scholar 

  7. 7.

    Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. 2011;71(s1):S52–63. https://doi.org/10.1111/j.1752-7325.2011.00233.x.

    Article  PubMed  PubMed Central  Google Scholar 

  8. 8.

    Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. Bmj. 2015;350:h1258. DOI. https://doi.org/10.1136/bmj.h1258.

  9. 9.

    Bearman SK, Weisz JR, Chorpita BF, Hoagwood K, Ward A, Ugueto AM, et al. More practice, less preach? The role of supervision processes and therapist characteristics in EBP implementation. Adm Policy Ment Health Ment Health Serv Res. 2013;40(6):518–29. https://doi.org/10.1007/s10488-013-0485-5.

    Article  Google Scholar 

  10. 10.

    Bond GR, Drake RE, McHugo GJ, Rapp CA, Whitley R. Strategies for improving fidelity in the national evidence-based practices project. Res Soc Work Pract. 2009;19(5):569–81. https://doi.org/10.1177/1049731509335531.

    Article  Google Scholar 

  11. 11.

    Boulton ER, Horne M, Todd C. Multiple influences on participating in physical activity in older age: developing a social ecological approach. Health Expect. 2018;21(1):239–48 https://doi.org/10.1111/hex.12608.

    PubMed  PubMed Central  Article  Google Scholar 

  12. 12.

    Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50 https://doi.org/10.1186/1748-5908-4-50.

    PubMed  PubMed Central  Article  Google Scholar 

  13. 13.

    Dugmore O, Orrell M, Spector A. Qualitative studies of psychosocial interventions for dementia: a systematic review. Aging Mental Health. 2015(ahead-of-print):1–13. DOI: https://doi.org/10.1080/13607863.2015.1011079.

  14. 14.

    Lawrence V, Fossey J, Ballard C, Moniz-Cook E, Murray J. Improving quality of life for people with dementia in care homes: making psychosocial interventions work. Br J Psychiatry. 2012;201(5):344–51. https://doi.org/10.1192/bjp.bp.111.101402.

    Article  PubMed  Google Scholar 

  15. 15.

    Toomey E, Matthews J, Hurley DA. Using mixed methods to assess fidelity of delivery and its influencing factors in a complex self-management intervention for people with osteoarthritis and low back pain. BMJ Open. 2017;7(8):e015452. https://doi.org/10.1136/bmjopen-2016-015452.

    Article  PubMed  PubMed Central  Google Scholar 

  16. 16.

    Leung P, Yates L, Orgeta V, Hamidi F, Orrell M. The experiences of people with dementia and their carers participating in individual cognitive stimulation therapy. Int J Geriatric Psychiatry. 2017;32(12):e34–42. https://doi.org/10.1002/gps.4648.

    Article  Google Scholar 

  17. 17.

    Mahoney EK, Trudeau SA, Penyack SE, MacLeod CE. Challenges to intervention implementation: lessons learned in the bathing persons with Alzheimer's disease at home study. Nurs Res. 2006;55(2):S10–S6.

    PubMed  Article  Google Scholar 

  18. 18.

    Roberts SH, Bailey JE. Incentives and barriers to lifestyle interventions for people with severe mental illness: a narrative synthesis of quantitative, qualitative and mixed methods studies. J Adv Nurs. 2011;67(4):690–708. https://doi.org/10.1111/j.1365-2648.2010.05546.x.

    Article  PubMed  Google Scholar 

  19. 19.

    Cooper C, O'Cathain A, Hind D, Adamson J, Lawton J, Baird W. Conducting qualitative research within clinical trials units: avoiding potential pitfalls. Contemporary Clinical Trials. 2014;38(2):338–43 https://doi.org/10.1016/j.cct.2014.06.002.

    PubMed  Article  Google Scholar 

  20. 20.

    McKenna JW, Flower A, Ciullo S. Measuring fidelity to improve intervention effectiveness. Interv Sch Clin. 2014;1053451214532348. https://doi.org/10.1177/1053451214532348.

  21. 21.

    Moncher FJ, Prinz RJ. Treatment fidelity in outcome studies. Clin Psychol Rev. 1991;11(3):247–66 https://doi.org/10.1016/0272-7358(91)90103-2.

    Article  Google Scholar 

  22. 22.

    Munafo MR, Smith GD. Repeating experiments is not enough. Nature. 2018;553(7689):399–401.

    CAS  PubMed  Article  Google Scholar 

  23. 23.

    Creswell JW. A concise introduction to mixed methods research. United States of America: SAGE publications; 2014.

  24. 24.

    Dahlgren G, Whitehead M. Policies and strategies to promote social equity in health. Stockholm: Institute for future studies. 1991. Retrieved from: https://www.researchgate.net/profile/Goeran_Dahlgren/publication/5095964_Policies_and_strategies_to_promote_social_equity_in_health_Background_document_to_WHO_-_Strategy_paper_for_Europe/links/569540f808aeab58a9a4d946.pdf.

  25. 25.

    Michie S, Atkins L, West R. The behaviour change wheel: a guide to designing interventions. Great Britain: Silverback Publishing; 2014.

    Google Scholar 

  26. 26.

    Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011a;6(1):42 https://doi.org/10.1186/1748-5908-6-42.

    PubMed  PubMed Central  Article  Google Scholar 

  27. 27.

    Steinmo S, Fuller C, Stone S, Michie S. Characterising an implementation intervention in terms of behaviour change techniques and theory: the 'Sepsis Six' clinical care bundle. Implement Sci. 2015;10(1):111 https://doi.org/10.1186/s13012-015-0300-7.

    PubMed  PubMed Central  Article  Google Scholar 

  28. 28.

    Barker F, Atkins L, de Lusignan S. Applying the COM-B behaviour model and behaviour change wheel to develop an intervention to improve hearing-aid use in adult auditory rehabilitation. Int J Audiol. 2016;55(sup3):S90–S8 https://doi.org/10.3109/14992027.2015.1120894.

    PubMed  Article  Google Scholar 

  29. 29.

    Connell L, McMahon N, Redfern J, Watkins C, Eng J. Development of a behaviour change intervention to increase upper limb exercise in stroke rehabilitation. Implement Sci. 2015;10(1):34 https://doi.org/10.1186/s13012-015-0223-3.

    PubMed  PubMed Central  Article  Google Scholar 

  30. 30.

    Fulton EA, Brown KE, Kwah KL, Wild S, editors. StopApp: using the behaviour change wheel to develop an app to increase uptake and attendance at NHS Stop Smoking Services. Healthcare; 2016: Multidisciplinary digital publishing institute. DOI: https://doi.org/10.3390/healthcare4020031.

  31. 31.

    Toomey E, Matthews J, Guerin S, Hurley DA. Development of a feasible implementation Fidelity protocol within a complex physical therapy–led self-management intervention. Phys Ther. 2016;96(8):1287–98 https://doi.org/10.2522/ptj.20150446.

    PubMed  Article  Google Scholar 

  32. 32.

    Williams SL, McSharry J, Taylor C, Dale J, Michie S, French DP. Translating a walking intervention for health professional delivery within primary care: a mixed-methods treatment fidelity assessment. Br J Health Psychol. 2019; https://doi.org/10.1111/bjhp.12392.

  33. 33.

    Csipke E, Yates L, Moniz Cook E, Leung P, Charlesworth G, Walton H, Birt L, Orrell M. Promoting Independence in dementia: protocol for a feasibility trial of the PRIDE intervention for living well with dementia. Int J Clin Trials. 2018;5(4):177–85. https://doi.org/10.18203/2349-3259.ijct20184399.

    Article  Google Scholar 

  34. 34.

    Yates L, Csipke E, Moniz-Cook E, Leung P, Walton H, Charlesworth G, Spector A, Hogervorst E, Mountain G, Orrell M. The development of the promoting Independence in dementia (PRIDE) intervention to enhance independence in dementia. Clin Interv Aging. 2019;14:1615–30.

    PubMed  PubMed Central  Article  Google Scholar 

  35. 35.

    Walton H, Spector A, Williamson M, Tombor I, Michie S. Developing quality fidelity and engagement measures for complex health interventions. Br J Health Psychol. 2019; https://doi.org/10.1111/bjhp.12394.

  36. 36.

    Walton, H. J. (2018). Evaluating the implementation of social interventions to improve independence in dementia. Doctoral thesis (Ph.D). Retrieved from UCL discovery: http://discovery.ucl.ac.uk/10064480/.

  37. 37.

    Schlosser R. On the importance of being earnest about treatment integrity. Augmentative and Alternative Communication. 2002;18(1):36–44. https://doi.org/10.1080/aac.18.1.36.44.

  38. 38.

    Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7(1):37. https://doi.org/10.1186/1748-5908-7-37.

    Article  PubMed  PubMed Central  Google Scholar 

  39. 39.

    Stevenson M, Savage B, Taylor BJ. Perception and communication of risk in decision making by persons with dementia. Dementia. 2017;1471301217704119 https://doi.org/10.1177/1471301217704119.

  40. 40.

    Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. https://doi.org/10.1191/1478088706qp063oa.

    Article  Google Scholar 

  41. 41.

    Joffe H. Thematic analysis. In: Harper D, Thompson AR, editors. Qualitative Research Methods in Mental Health and Psychotherapy: A guide for students and practitioners. 1st ed. West Sussex: Wiley; 2012. p. 209–23.

    Google Scholar 

  42. 42.

    Clarke V, Braun V. Using thematic analysis in counselling and psychotherapy research: a critical reflection. Couns Psychother Res. 2018;18(2):107–10 https://doi.org/10.1002/capr.12165.

    Article  Google Scholar 

  43. 43.

    Neale J. Iterative categorization (IC): a systematic technique for analysing qualitative data. Addiction. 2016;111(6):1096–106. https://doi.org/10.1111/add.13314.

    Article  PubMed  PubMed Central  Google Scholar 

  44. 44.

    Braun V, Clarke V. What can “thematic analysis” offer health and wellbeing researchers? Int J Qual Stud Health Well Being. 2014;9. https://doi.org/10.3402/qhw.v9.26152.

  45. 45.

    Cook KE. Reliability assessments in qualitative health promotion research. Health Promot Int. 2011;27(1):90–101. https://doi.org/10.1093/heapro/dar027.

    Article  PubMed  Google Scholar 

  46. 46.

    Barbour RS. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ. 2001;322(7294):1115–7 Retrieved from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1120242/pdf/1115.pdf.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  47. 47.

    Golafshani N. Understanding reliability and validity in qualitative research. Qual Rep. 2003;8(4):597–606 Retrieved from: http://nsuworks.nova.edu/tqr/vol8/iss4/6.

    Google Scholar 

  48. 48.

    Guest G, MacQueen KM, Namey EE. Validity and reliability (credibility and dependability) in qualitative research and data analysis. Applied thematic analysis. London: Sage publications; 2012. p. 79–106. https://doi.org/10.4135/9781483384436.

    Google Scholar 

  49. 49.

    Krefting L. Rigor in qualitative research: the assessment of trustworthiness. Am J Occup Ther. 1991;45(3):214–22. https://doi.org/10.5014/ajot.45.3.214.

    CAS  Article  PubMed  Google Scholar 

  50. 50.

    Pistrang N, Barker C. Varieties of qualitative research: a pragmatic approach to selecting methods. 2012. https://doi.org/10.1037/13620-001.

    Google Scholar 

  51. 51.

    Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013;46(1):81–95.

    PubMed  PubMed Central  Article  Google Scholar 

  52. 52.

    Surr CA, Gates C, Irving D, Oyebode J, Smith SJ, Parveen S, Drury M, Dennison A. Effective dementia education and training for the health and social care workforce: a systematic review of the literature. Rev Educ Res. 2017;87(5):966–1002 https://doi.org/10.3102/0034654317723305.

    PubMed  PubMed Central  Article  Google Scholar 

  53. 53.

    Brown J. Training needs assessment: a must for developing an effective training program. Public Personnel Management. 2002;31(4):569–78 https://doi.org/10.1177/009102600203100412.

    Article  Google Scholar 

  54. 54.

    Sawyer T, White M, Zaveri P, Chang T, Ades A, French H, Anderson J, Auerbach M, Johnston L, Kessler D. Learn, see, practice, prove, do, maintain: an evidence-based pedagogical framework for procedural skill training in medicine. Acad Med. 2015;90(8):1025–33. https://doi.org/10.1097/ACM.0000000000000734.

    Article  PubMed  Google Scholar 

  55. 55.

    Kitwood TM. Dementia reconsidered: The person comes first. Buckingham: Open university press; 1997. p. 7–8.

    Google Scholar 

  56. 56.

    French SD, Green SE, Francis JJ, Buchbinder R, O'Connor DA, Grimshaw JM, et al. Evaluation of the fidelity of an interactive face-to-face educational intervention to improve general practitioner management of back pain. BMJ Open. 2015;5(7):e007886 https://doi.org/10.1136/bmjopen-2015-007886.

    PubMed  PubMed Central  Article  Google Scholar 

  57. 57.

    French S, Green S, O'Connor D, McKenzie J, Francis J, Michie S, et al. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the theoretical domains framework. Implement Sci. 2012;7(1):38 https://doi.org/10.1186/1748-5908-7-38.

    PubMed  PubMed Central  Article  Google Scholar 

  58. 58.

    Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, et al. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH behavior change consortium. Health Psychol. 2004;23(5):443.

    PubMed  Article  Google Scholar 

  59. 59.

    Hauer K, Schwenk M, Zieschang T, Essig M, Becker C, Oster P. Physical training improves motor performance in people with dementia: a randomized controlled trial. J Am Geriatr Soc. 2012;60(1):8–15 https://doi.org/10.1111/j.1532-5415.2011.03778.x.

    PubMed  Article  Google Scholar 

  60. 60.

    Fetherstonhaugh D, Tarzia L, Nay R. Being central to decision making means I am still here!: the essence of decision making for people with dementia. J Aging Stud. 2013;27(2):143–50 https://doi.org/10.1016/j.jaging.2012.12.007.

    PubMed  Article  Google Scholar 

  61. 61.

    Nyman SR, Adamczewska N, Howlett N. Systematic review of behaviour change techniques to promote participation in physical activity among people with dementia. Br J Health Psychol. 2018;23(1):148–70 https://doi.org/10.1111/bjhp.12279.

    PubMed  Article  Google Scholar 

  62. 62.

    Heyn P. The effect of a multisensory exercise program on engagement, behavior, and selected physiological indexes in persons with dementia. Am J Alzheimer’s Dis Other Dementias®. 2003;18(4):247–51 https://doi.org/10.1177/153331750301800409.

    Article  Google Scholar 

  63. 63.

    Cevasco AM, Grant RE. Comparison of different methods for eliciting exercise-to-music for clients with Alzheimer's disease. J Music Ther. 2003;40(1):41–56 https://doi.org/10.1093/jmt/40.1.41.

    PubMed  Article  Google Scholar 

  64. 64.

    Chard G, Liu L, Mulholland S. Verbal cueing and environmental modifications: strategies to improve engagement in occupations in persons with Alzheimer disease. Physical Occup Therapy Geriatrics. 2009;27(3):197–211 https://doi.org/10.1080/02703180802206280.

    Article  Google Scholar 

  65. 65.

    Cook C, Fay S, Rockwood K. Decreased initiation of usual activities in people with mild-to-moderate Alzheimer's disease: a descriptive analysis from the VISTA clinical trial. Int Psychogeriatr. 2008;20(5):952–63 https://doi.org/10.1017/S1041610208007230.

    PubMed  Article  Google Scholar 

  66. 66.

    Dooley NR, Hinojosa J. Improving quality of life for persons with Alzheimer’s disease and their family caregivers: brief occupational therapy intervention. Am J Occup Ther. 2004;58(5):561–9. https://doi.org/10.5014/ajot.58.5.561.

    Article  PubMed  Google Scholar 

  67. 67.

    Padilla R. Effectiveness of interventions designed to modify the activity demands of the occupations of self-care and leisure for people with Alzheimer’s disease and related dementias. Am J Occup Ther. 2011;65(5):523–31. https://doi.org/10.5014/ajot.2011.002618.

    Article  PubMed  Google Scholar 

  68. 68.

    Spector A, Thorgrimsen L, Woods B, Royan L, Davies S, Butterworth M, et al. Efficacy of an evidence-based cognitive stimulation therapy programme for people with dementia. Br J Psychiatry. 2003;183(3):248–54. https://doi.org/10.1192/bjp.183.3.248.

  69. 69.

    Michie S, Abraham C, Eccles MP, Francis JJ, Hardeman W, Johnston M. Strengthening evaluation and implementation by specifying components of behaviour change interventions: a study protocol. Implement Sci. 2011b;6(1):10 https://doi.org/10.1186/1748-5908-6-10.

    PubMed  PubMed Central  Article  Google Scholar 

  70. 70.

    Bunn F, Dickinson A, Barnett-Page E, Mcinnes E, Horton K. A systematic review of older people's perceptions of facilitators and barriers to participation in falls-prevention interventions. Ageing Soc. 2008;28(04):449–72. https://doi.org/10.1017/S0144686X07006861.

    Article  Google Scholar 

  71. 71.

    Rimmer JH, Riley B, Wang E, Rauworth A, Jurkowski J. Physical activity participation among persons with disabilities: barriers and facilitators. Am J Prev Med. 2004;26(5):419–25. https://doi.org/10.1016/j.amepre.2004.02.002.

    Article  PubMed  PubMed Central  Google Scholar 

  72. 72.

    Ong LM, De Haes JC, Hoos AM, Lammes FB. Doctor-patient communication: a review of the literature. Soc Sci Med. 1995;40(7):903–18 https://doi.org/10.1016/0277-9536(94)00155-M.

    CAS  PubMed  Article  Google Scholar 

  73. 73.

    Safeer RS, Keenan J. Health literacy: the gap between physicians and patients. Am Fam Physician. 2005;72(3):463–8.

    PubMed  Google Scholar 

  74. 74.

    Williams MV, Davis T, Parker RM, Weiss BD. The role of health literacy in patient-physician communication. Family Med-Kansas City. 2002;34(5):383–9.

    Google Scholar 

Download references

Acknowledgements

Thank you to Dr. Emese Csipke and the wider PRIDE team for their support throughout the project. Thank you to Dr. Danielle D’Lima, Dr. Ailbhe Finnerty, Dr. Niall Anderson, Catherine Lawrence, Paulina Schenk, Silje Zink and Emily Hayes for providing feedback on an earlier version of this manuscript.

Funding

Holly Walton’s PhD research was funded by the Economic and Social Research Council (ESRC) Doctoral Training Centre (Grant reference: ES/J500185/1), as part of the Promoting Independence in Dementia programme (Grant references: 2014–2015 ES/L001802/1; 2015–2019 ES/L001802/2). The funding bodies played no role in designing, conducting, analysing, interpreting or reporting the results of the research.

Author information

Affiliations

Authors

Contributions

HW, with support from SM, AS, and IT designed, planned and led the mixed-methods study described in this manuscript. HW developed the PRIDE framework and fidelity checklists with support from SM, AS and IT. Members of the PRIDE team gave feedback on draft fidelity checklists and helped to gain feedback from PPI members. PRIDE researchers helped to store the data and researchers at individual sites helped to keep track of the data. Audio-recordings were transcribed by a professional transcription service. HW anonymised transcripts. HW and JB piloted coding guidelines for fidelity assessment. HW coded intervention sessions for fidelity and MW double-coded a percentage of data. HW conducted the analysis with input from all authors. For the qualitative interviews: HW designed and developed the interview schedules with support from SM, AS and IT. Audio-recordings were transcribed by a professional transcription service. HW conducted the thematic analysis. JB double coded a percentage of transcripts to test the coding frames and reviewed the themes and example extracts. HW and AR coded extracts according to the COM-B model. SM, AS and IT reviewed the themes and analysis. To develop recommendations, HW mapped the findings onto the Behaviour Change Wheel. HW drafted the manuscript and all authors commented on, and approved the final draft.

Corresponding author

Correspondence to Holly Walton.

Ethics declarations

Ethics approval and consent to participate

The PRIDE programme of research has received Health Research Authority approval and NHS ethical approval as part of the feasibility trials intervention ethics application (NHS East Midlands – Nottingham 1 Research Ethics committee, REC reference number: 16/EM/0044). For Stage 1, participants provided written informed consent for audio-recording of intervention sessions as part of their written consent to take part in the PRIDE intervention. For Stage 2 of this study, all providers, people with dementia and supporters gave written informed consent to take part in the interviews.

Consent for publication

Not applicable as all individual participant quotes are fully anonymised. As part of the consent to participate, all participants gave consent to their anonymised quotes being used in reports of the research.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Walton, H., Spector, A., Roberts, A. et al. Developing strategies to improve fidelity of delivery of, and engagement with, a complex intervention to improve independence in dementia: a mixed methods study. BMC Med Res Methodol 20, 153 (2020). https://doi.org/10.1186/s12874-020-01006-x

Download citation

Keywords

  • Fidelity of delivery
  • Engagement
  • Behaviour change
  • Dementia
  • Qualitative
  • Observation
  • Mixed methods
  • Implementation
  • Complex health intervention