Skip to main content

The value of social practice theory for implementation science: learning from a theory-based mixed methods process evaluation of a randomised controlled trial

Abstract

Background

Although there is trial evidence that complex interventions are effective for the self-management of heart failure, little evidence supports their effectiveness in routine practice. We used Social Practice Theory to guide a Type 1 Hybrid Trial: a mixed methods process evaluation of a complex intervention for heart failure. The objective of this paper is to explore the value of Social Practice Theory for implementation science.

Methods

Social Practice Theory informed a mixed methods process evaluation of a multi-centre randomised controlled trial of a 12 week home-based intervention to optimise self-care support for people with heart failure and their caregivers - Rehabilitation EnAblement in Chronic Heart Failure (REACH-HF). Interviews were conducted with 19 people with heart failure and 17 caregivers at 4 months and 12 months after recruitment into the trial. Cases were constructed at the level of the individual, couple, facilitator and centre; and included multi-modal process and outcome data. Evaluative coding and subsequent within- and cross-case analyses enabled the development of a typology of relationships linking fidelity of intervention delivery and tailoring of content to individual needs and concerns. Social Practice Theory was used to interrogate the relationships between elements of the intervention and their implementation.

Results

Of 216 trial participants, 107 were randomised to the intervention (REACH-HF plus usual care). The intervention was most effective when fidelity was high and delivery was tailored to the individual’s needs, but less effective when both tailoring and fidelity were low. Theory-based analysis enabled us to model complex relationships between intervention elements (competencies, materials and meanings) and social context. The findings illustrate how intervention fidelity and tailoring are contextual and how the effectiveness of the REACH-HF intervention depended on both optimal alignment and implementation of these elements.

Conclusion

The study demonstrates the utility of theory-based analysis which integrates data from multiple sources to highlight contexts and circumstances in which interventions work best. Social Practice Theory provides a framework for guiding and analysing the processes by which a complex intervention is evaluated in a clinical trial, and has the potential to guide context-specific implementation strategies for clinical practice.

Trial registration

ISRCTN, IISRCTN86234930. Registered 13th November 2014.

Peer Review reports

Background

Although there is evidence that complex interventions are effective for the self-management of heart failure, many are not implemented in routine practice. Exercise based cardiac rehabilitation in patients with heart failure have shown improvements in health-related quality of life and a reduction in hospitalisations [1]. Whilst evidence for the added value of centre and group-based cardiac rehabilitation for patients with heart failure is strong, uncertainty remains about the effectiveness of alternative modes of delivery [2].

Previous research has failed to describe the complexity of living with heart failure and the burden of self-management and adherence to intervention regimens [3]; failed to identify elements which consistently improve outcomes in self-management interventions for people with heart failure [4]; and failed to explain how programme elements of cardiac rehabilitation interventions interact [5]. Systematic reviews of randomised controlled trials for heart failure have also identified a lack of process evaluations assessing change processes which may inform wider implementation [6].

The Rehabilitation Enablement in Chronic Heart Failure (REACH-HF) multicentre randomised controlled trial demonstrated the clinical effectiveness and cost-effectiveness of the REACH-HF cardiac rehabilitation and self-care intervention for patients with heart failure with reduced ejection fraction (HFrEF) and their caregivers [1, 7, 8]. The randomised controlled trial subsequently demonstrated a statistically significant and clinically meaningful between-group (intervention-control) difference in the Minnesota Living with Heart Failure Questionnaire (MLHFQ) score of − 5.7 points (95% confidence interval − 10.6 to − 0.7) in favour of the REACH-HF intervention group (p = 0.025) at 12 months (minimal clinically important difference (MCID) for MLFHQ is 5 points; a higher score indicates a worse clinical outcome) [7].

In this paper we report the results of the theory-based process evaluation and identify the extent to which Social Practice Theory has the potential to progress our understanding from a pragmatic trial of a clinical intervention at T2 (translation to patients), to the development of an intervention strategy for implementation at T3 (translation to practice) [9, 10], e.g. via the identification of configurations of intervention elements [11, 12].

While the MRC framework for process evaluations of complex interventions remains a valuable tool for exploring mechanisms of impact, debates regarding the role of mechanisms in complex interventions and adaptation to context remain unresolved [13]. Trialists have previously called for greater theorising and understanding of the social aspects of complex interventions [14,15,16]. Paying attention to social context can improve the effectiveness of interventions in a given context; promote sustainability of an intervention in that context; and promote dissemination in other contexts [17, 18]. More specifically, self-management interventions for chronic conditions could be considerably more effective if the experiences, social and contextual interactions and sense-making processes of both patients and health professionals were prioritised in trial design, evaluation and implementation.

Normalisation Process Theory (NPT) has provided a mid-range theory to guide the implementation and evaluation of complex interventions [19], but the enduring gap between research and implementation suggests that identification of other potential solutions is warranted [20]. While the focus of the NPT framework is on the work that professionals do to ensure that interventions become normalised, less attention has been given to the identification and evaluation of the elements that constitute the mechanisms of impact and how they may or may not operate in a given context [21]. Braithwaite et al. argue that a linear pipeline model of implementation demotes contextual characteristics to ‘confounders’, rather than viewing them as the normal conditions of the practice of healthcare [22]. This perspective is supported by several authors, who have reported on the proliferation of behaviour change interventions that focus mainly on intra-individual processes of change, while identifying that processes related to social context have been overlooked [23, 24].

Social practice theory

One way to operationalise the evaluation of an intervention ‘in context’ is via Social Practice Theory. This theory has proven popular in environmental research, where it has identified the systemic failure of interventions to change behaviour [25, 26]. In contrast to linear models of implementation, Social Practice Theory offers a more dynamic framework because, rather than focusing on individuals or institutions or programme theories, it is concerned with the implementation of ‘practice bundles’ or sets of interconnected elements and the ways in which practices are reproduced, maintained, stabilised, challenged and surpassed [25, 27].

Social practice is constituted by interdependencies between diverse elements including bodily activities (e.g. exercising), mental activities (e.g. undertaking crosswords for relaxation), material objects (e.g. use of scales for weighing) and their use, background knowledge (e.g. from lay and professional carers) in the form of shared understandings, states of emotion (e.g. anxiety) and motivational knowledge (e.g. the value of maintaining social activities) [11]. Practices have 3 elements: 1) Materials including technologies, tangible, physical entities, and the stuff of which objects are made; 2) Competences which encompasses skills, knowhow and techniques; and 3) Meanings which include symbolic or shared meanings, social norms, ideas and collective aspirations. These elements are the ‘building blocks of practice’, which allow us to describe processes of transformation, diffusion and circulation [11]. For practices to be sustainable they need to capture and retain practitioners who are willing and able to enact these integrating processes and who are therefore willing and able to keep the practices alive. The value of this approach is that it allows us to map the uneven landscapes of opportunity and unequal patterns of access between participants; enabling us to identify the relatively privileged (with the physical, cognitive or socio-economic means to engage), and the marginalised (i.e. the socially excluded who are lacking opportunities to become ‘carriers of practice’) [28, 29].

Brief description of the REACH-HF trial

The aim of the Rehabilitation Enablement in Chronic Heart Failure (REACH-HF) programme of research was to develop and evaluate a 12-week, home-based facilitated heart failure manual to enhance the quality of life and self-care of people with heart failure and their caregivers (Table 1) [1]. The support needs of people with heart failure and their caregivers were established, and the REACH-HF intervention developed by an intervention mapping process and feasibility study [31].

Table 1 REACH-HF components

The REACH-HF randomised controlled trial was conducted at 4 centres in the UK, and a theory-based process evaluation undertaken to assess fidelity of delivery (using a checklist applied to recordings of facilitator-patient-caregiver interactions), and to characterise patients’ and caregivers’ observed and self-reported responses to the intervention [7, 32]. ‘The aim of this paper is to use Social Practice Theory to identify mechanisms through which the intervention did or did not work, and consider the implications for subsequent implementation in routine clinical practice.

Contributions to the literature

  • We undertook a Type 1 Hybrid Trial (a randomised controlled trial evaluating the effect of a complex intervention for the self-management of heart failure, and simultaneously collected longitudinal process data about how the intervention was implemented) and used a rigorous mixed methods approach to explore data from multiple sources.

  • Social Practice Theory was used to map and interrogate the relationships between the various elements (materials, competencies and meanings) of the intervention and their implementation in a range of clinical contexts over time.

  • Social Practice Theory enabled us to identify that the REACH-HF intervention was optimised when the competencies of the facilitator and the intervention materials were tailored to the goals and needs of the participant in ways which enabled participants to transform their meaning and develop sustainable and flexible self-management practices - but only where the magnitude of any complex problems were within the scope of the intervention.

  • Our findings suggest that Social Practice Theory has the potential to progress our understanding from a trial of a clinical intervention at T2 (translation to patients), to guide the development of intervention strategies for implementation at T3 (translation to practice) via the identification of the most appropriate configurations of bundles of intervention elements for implementation of personalised interventions at scale.

Methods

Study design

We conducted a theory-based process evaluation of a randomised controlled trial evaluating the effect of the REACH-HF intervention for the self-management of heart failure [1]. We employed a Hybrid Type 1 trial design which both tested the effectiveness of the evidence based intervention and collected data on the implementation processes [12]. The methods of both the REACH-HF randomised controlled trial and process evaluation have been described in detail [7, 32].

Two hundred sixteen patients and 97 caregivers were recruited to the randomised controlled trial from 4 geographical regions across the UK (Birmingham, Cornwall, Gwent and York). One hundred seven patients were randomised to the intervention arm of the trial, and 109 to usual care. The sampling frame for the process evaluation was the intervention arm of the randomised controlled trial, from which 19 patients were sampled for maximum variation in their baseline scores for the primary and secondary outcome measures of the trial: The primary outcome was disease-specific health-related quality of life at 12 months measured using the Minnesota Living with Heart Failure Questionnaire (MLHFQ) [33]. Secondary outcomes were death, hospitalization, generic quality of life (five-dimension EuroQol (EQ-5D-5L) scale) [34], psychological wellbeing (Hospital Anxiety and Depression Scale (HADS)) [35], exercise capacity (incremental shuttle walk test) [36] and physical activity assessed using a GeneActiv accelerometer [37]. Additional secondary measures included the HeartQoL questionnaire [38] and Self-Care of Heart Failure Index [39]. At the baseline clinic visit sociodemographic data and information on past medical history from primary and secondary care records were collected, including New York Heart Association classification [40], key comorbidities, concomitant cardiac drugs and presence of implantable cardiac devices.

Data collection

Both patients and caregivers were contacted by the interviewer and, with their consent, interviewed 4 months and 12 months after the baseline study enrolment visit. Face to face or telephone interviews were conducted by experienced qualitative researchers (JW, LM) and, with participant consent, audio-recorded and transcribed. A semi-structured interview schedule enabled the interviewers to explore existing self-care practices, perceptions of the interaction with the facilitator and the extent to which different aspects of the intervention were engaged with. As well as assessing behaviour change, we also sought to explore contextual factors which may have facilitated or inhibited change, and the experience of living with progressive chronic heart failure more generally, and change over time.

For the 19 people with heart failure and 17 caregivers who constituted our sampling frame (2–3 patients per facilitator across the 4 regions), all face-to face meetings with the facilitator (typically one 90 min and two to three 45 min), and subsequent telephone contacts (typically 3–6 per patient) were audio-recorded by the facilitator. A 13-domain fidelity checklist was developed and piloted during the feasibility study, and used in the process evaluation to assess the facilitators’ delivery of the intervention. The intervention designers (JW, CG) applied the checklist to the audio recorded intervention sessions [31]. Fidelity scoring attributed a numerical value (0–6) for each of the 13 domains on the fidelity checklist. A purposive sample of audio recordings were used to identify examples of optimal and sub-optimal delivery fidelity (chosen on the basis of maximum diversity of fidelity scores between patients, and where dissonance was identified by JW and CG). Ethical approval was granted by the North West- Lancaster Research Ethics Committee; the trial registration number is ISRCTN86234930.

Data analysis

Multi-modal data from both the randomised controlled trial and process evaluation were managed with Stata v14.2 [41] and Nvivo 11 Pro [42] and assigned to cases at four levels: all data for a patient/caregiver; dyads of patients and caregivers; all data for each facilitator; and each centre. Multi-modal cases were then created for each patient/caregiver [43], including interview transcripts, contact sheets, audio recordings of facilitator-patient sessions and associated fidelity scores, summaries of interactions, field notes, clinical data such as case-report forms, and patient reported outcome measures.

Interview transcripts were subject to a preliminary cycle of evaluative coding, which assigned a judgement about the value or significance of the intervention [44]. This noted the presence or absence of an element (e.g. chair based or walking exercise) and the extent to which it was positively or negatively evaluated by the patient/caregiver. This amplified the fidelity scores, which attributed a numerical value (0–6) to the facilitation as outlined in the facilitator training manual. A secondary cycle of explanatory coding extended the mapping of variance across the dataset, and enabled within-case analyses over time, cross-case analyses of intervention elements and social and contextual variables [44], and the further identification of directional processes [45]. Fidelity was coded on the basis of the emphasis given to the intended core components of the intervention, while tailoring was coded in relation to the spirit of the intervention being adapted to the local context in order to achieve a ‘good ecological fit’ [13].

Social Practice Theory was used to further interrogate the relationships between the various elements of the intervention [11]. While JF coded the data, the iterative analysis was undertaken through extensive team discussions with JW who had collected most of the data, produced extensive field notes, reviewed the transcripts and assessed the audio-recorded facilitation sessions for fidelity, and NB who independently annotated the content of purposively sampled facilitated sessions. Crucially, the qualitative data collection and preliminary analysis was completed before the overall trial outcome data were known, to minimise bias in interpretation [16, 43]. Analysing the longitudinal data from all 19 cases over two time-points [43, 45] enabled the development of a typology of relationships exploring fidelity of intervention delivery and tailoring of content to individual needs and concerns..

Results

Repeat semi-structured interviews were conducted with 19 patients (14 at 12 months) and 17 caregivers (15 at 12 months) between August 2015 and August 2016. Baseline demographic and health characteristics of the 107 REACH-HF patient participants in the intervention arm are set out in Table 2, with characteristics of the 19 interviewed participants shown for descriptive comparison. Data has been anonymised.

Table 2 Baseline characteristics of patients with Heart Failure participating in the Process Evaluation

Our in-depth analysis of this multi-modal data enabled us to create a 2 × 2 typology of fidelity and tailoring [46]. Here we provide four in depth illustrative cases (Table 3) that represent each of the possible combinations of fidelity and tailoring, and which illuminate the strengths and weaknesses of the REACH-HF intervention when viewed from a social practice perspective. These cases were selected for comparative purposes and each typifies one quadrant of the table, and demonstrate the results of our theory-based analysis (Table 4). High fidelity and good tailoring enabled facilitators to improve patients’ competencies, develop appropriate meanings and provide suitable materials. Conversely, poor fidelity and/or poor tailoring meant that some or all of patients’ competencies, meanings or materials were not enhanced, particularly in situations of little social and material capital. Not all cases were clear cut.

Table 3 Illustrative cases: Baseline Data
Table 4 Typology of Fidelity and Tailoring: Longitudinal data

Good fidelity and good tailoring

Mary is a 74 year old woman, diagnosed with heart failure 3 years before the study, who lives with her husband but is cared for by her three daughters. Mary received the lowest ‘dose’ of REACH-HF in these examples (293 min, over three face-to-face and three telephone intervention sessions). At the initial assessment, the facilitator explored Mary’s current understanding of her situation and baseline ‘capacity’ before gradually introducing the REACH-HF manual. This is reflected in the high fidelity scores at the initial intervention session (5 or more in 7 of the 13 domains) and subsequently (4–5 in most domains). At the initial assessment, the facilitator linked the discussion of Mary’s arthritis to her capacity for walking before going on to talk about the exercise component of the manual (e.g. the participant’s existing meaning and competence, and trial materials). This assessment was used to elicit Mary’s highest-priority self-care goal (walking) and the work required to fulfil this. The facilitator provided answers to Mary’s questions (thus improving her competence), as well as providing explicit instances of information to augment Mary’s understanding of her condition. The information provided by the facilitator was reinforced with the tailored use of the manual, which enabled Mary to create new understanding and meaning around her heart failure.

At 12 months Mary was clear that her enduring use of the manual after the intervention period improved her self-care. To sustain change over time, the facilitator drew on Mary’s existing relationships with her heart failure specialist nurse (HFSN) and General Practitioner (GP), and signposted her to their continued support. As well as drawing on the competencies of other professionals, this skilled facilitator was also able to transform family support, in response to Mary’s suggestion that she was feeling stifled by her daughters. Subsequently Mary’s daughters worked with the facilitator and the Friends and Family resource to support Mary in reaching her goals. Mary’s daughter annotated the Progress Tracker with stickers to help navigate key tasks, as well as developing a one-page tailored ‘tracker’ for Mary to write down her exercise, signs and symptoms, relaxation activities and sleep pattern after the intervention period. This enabled Mary to continue learning and develop her new found autonomy as her walking and confidence improved.

By using the initial assessment session to take a comprehensive medical history and explore Mary’s goals, this facilitator was able to use her time most efficiently and tailor the intervention to Mary’s needs. The facilitator also engaged her social support network to involve a series of others in working with Mary to achieve her goal and meet her needs. By working with Mary’s existing social resources (both lay and professional) this facilitator enabled self-care practices (e.g., walking) and social support systems to develop, which changed the meaning that Mary attributed to living with heart failure and enhanced her coping competencies.

Good fidelity and poor tailoring

While the intervention was tailored to Mary’s needs with good effect, John’s enduring socio-economic circumstances inhibited the effectiveness of the intervention. John is a 71 year old divorced man with depression, who smokes cigarettes and was diagnosed with heart failure 6 months previously. Living alone in a privately rented first-floor flat, but about to move into ground-floor social housing, he was troubled by large debts to utility companies. Despite suggesting that he had lived with heart failure since having rheumatic fever at 8 years old, John demonstrated that he did not understand how his heart worked to support activity.

Although the facilitator took a medical history and introduced the elements of the intervention, John spent much time talking about his stressful financial situation. Running out of time, the facilitator was unable to get beyond John’s social problems and did not elicit a self-care goal from John to underpin his rehabilitation. Thereafter the conversation remained facilitator-led, with little evidence of rapport building. Instead the overarching concern remained debt and over time tailoring was diminished as the facilitator adopted a more didactic style. Towards the end of this initial assessment session, the facilitator drew on her professional repertoire and provided John with a plumbing metaphor to explain how his cardiovascular system works, but while she proceeded with her explanation John said “you’re rattling on now” and she did not tailor the session to his response. This is reflected in the facilitator notes for this session which detailed her interpretation of the session rather than John’s.

Throughout the intervention period John continued to record his mood as ‘good’ in the Progress Tracker despite his enduring problems, and he provided minimal details about his exercise and medication use, other than noting that he gained weight. John subsequently withdrew from the randomised controlled trial and process evaluation, but the focus when interviewed at 4 months was about a perceived better future, rather than present, and his motivation for cutting down smoking was due to planned accommodation changes, rather than any new meaning attributed to REACH-HF.

While Mary’s facilitator engaged Mary’s existing support network to supplement her behaviour change, John has little social or material capital to draw on. His underlying depression and perception of his current circumstances are such that he is unable to engage with the new material and competencies that REACH-HF provide in a way that would enable him to construct a new meaning or self-care practices around his heart failure. Although the facilitator fully acknowledged his distressing circumstances neither her existing competencies nor the intervention materials could be tailored to mitigate such extensive socio-economic deprivation.

Poor fidelity and good tailoring

Sixty-eight year old Helen received one of the highest doses of REACH- HF at 470 min, over 4 face-to-face and 7 telephone sessions with her facilitator. She was recently diagnosed with heart failure and a lung ‘nodule’, and has long standing anxiety and depression, with some of the highest HADS scores (Anxiety = 19, Depression = 11) in the wider trial population at baseline. Helen lives with her husband who is her caregiver, but who struggles to deal with her emotional state. Unlike John, Helen had many interactions with multiple health professionals in the 6 months prior to the trial. Helen’s facilitator began by dealing with Helen’s concerns head-on: unpacking her fear of dying. This is one of the few cases in which the facilitator demonstrated the required exercise level to the patient, by actively walking with her and demonstrating warming up, cooling down, and the required effort for exercising the heart, which Helen valued.

However, while Helen was happy to do the physical exercise that the facilitator suggested, she did not fully engage with the heart failure manual – hiding it from her family and ignoring the sections which would have increased her own learning about the severity of her condition. By failing to increase her knowledge and understanding about heart failure, Helen learned by rote and was very proud of recording her 1900 steps a day in the Progress Tracker, which reflected the focus of the facilitated sessions.

While improving her exercise capacity was beneficial, this was in the absence of wider learning about her condition which meant that over time 1900 steps became the end rather than the means of improving her quality of life. The facilitator was viewed as kind and caring, but this was to the detriment of challenging Helen’s fears or supporting her autonomy. The subsequent sessions were increasingly didactic, as the facilitator failed to elicit the required response from Helen and instead made assumptions on her behalf. For example in the second audio-recorded session the facilitator asked Helen what her priorities were, but failed to wait for Helen’s reply. Subsequently other missed opportunities for engagement included not reviewing Helen’s physical symptoms and only using the heart failure manual in a responsive way, so that relevant sections were repeatedly omitted. In the final audio-recorded session the facilitator did not check Helen’s understanding of heart failure or explore why she had not contacted a local agency; rather the facilitator took on the task of arranging appointments and medication.

Helen achieved a limited transformation as a result of her involvement in REACH-HF. While the facilitator demonstrated a degree of competence in modelling the required exercise, Helen assumed that the circuit that they covered was sufficient and there was limited use of additional REACH-HF materials. Helen learned that her heart needed to be exercised, but this was achieved via habitual repetition rather than any increased competence or new meaning that would have allowed Helen to tailor her exercise to the weather or extend it to improve her mental health. Furthermore, many of her existing behaviours around denial and lack of agency went unchallenged.

Poor fidelity and poor tailoring

The final case is Dorothy, a 73 year old widow who lives alone and often looks after her grandchildren. She was diagnosed with heart disease 6 months before the trial, and while struggling to come to terms with her diagnosis was one of the sickest patients in the process evaluation, with an ejection fraction of only 21% at baseline cardiac assessment.

This example is striking for what was absent. The audio-recording of the initial session began with the introduction of the REACH-HF manual, and there was no discussion of Dorothy’s medical history, goals, or social situation. Here, fidelity scores were low at baseline and subsequently (all under 2.5 and deemed inadequate). The facilitator neither elicited or addressed Dorothy’s concerns despite her being newly diagnosed, having seen no health professionals since diagnosis, and having a limited understanding of her condition.

This also seems to be an example of lack of professional competence, as the facilitator was unable to answer adequately a straightforward question about Dorothy’s heart failure. While Dorothy had little understanding of her heart failure, the facilitator lacked the competence to improve her learning, as demonstrated in a subsequent session when Dorothy asked a question about medication and the facilitator neither found the relevant section in the manual nor drew upon any wider professional knowledge. Because the facilitator continually failed to engage Dorothy in the programme, Dorothy gained little understanding of the utility of the trial materials, and suggested that she had little faith in the facilitator’s competence. Post- intervention Dorothy was still struggling to reconcile that she had a chronic condition that requires self-care. She admitted that she would let her weight increase by seven to fourteen pounds (potentially fatal) before she would call for professional help.

Dorothy lacked the skills and competence to care for her heart failure, and having a facilitator who failed to challenge her existing interpretations ensured that Dorothy did not engage with the REACH- HF intervention. Like Helen, Dorothy now has a walk that she undertakes regularly, but she has been unable to develop any meaningful notion of her heart failure, as she still attributes her lack of deterioration to the lack of severity of her heart failure, whereas paradoxically any subsequent deterioration would be attributed to “just my age”.

Discussion

Summary of key findings

This is the first process evaluation of an evidence-based home-based rehabilitation intervention to optimise self-care support for heart failure patients and their caregivers. We identified that the intervention was most effective when fidelity was high and delivery was tailored to the individual’s needs, and that the intervention was less effective when both tailoring and fidelity were low. Furthermore, our theory-based analysis enabled us to model complex relationships between intervention elements (competencies, materials and meanings) and social context. For example, in our ‘best case’ the meaning that Mary attributed to living with heart failure was transformed by the strategic use of professional competencies (skilled goal elicitation), purposeful use of materials (the tailored use of the manual and progress tracker), and the augmentation of meanings (through the continued engagement and support of Mary and her family). Whereas, in our ‘worst case’ Dorothy’s meaning was not transformed because neither the competencies of the facilitator (both heart failure specific and wider professional knowledge) or materials (which were not tailored to needs) were able to engage her in the programme. By using Social Practice Theory to interrogate these relationships between the various elements of the intervention over time we were able to identify how fidelity and tailoring are contextual and how effectiveness of the REACH-HF intervention was dependent on both optimal alignment and implementation of elements. Therefore Social Practice Theory not only progressed our understanding of both the findings and processes of a trial of a clinical intervention but also provided insights to inform the development of strategies for implementation in routine practice [20, 47].

Interpretation and implications of results

The constituents of the REACH-HF intervention were chosen in response to the challenges of implementing a rehabilitation intervention within the patient’s home [48], based on empirical evidence of their effectiveness, and engagement with stakeholders [31]. While theoretically informed qualitative analyses have previously identified the importance of self-care ‘assemblages’ in routine practice [49, 50], we sought to explore the dynamic interactions of fidelity and tailoring during the delivery of the REACH-HF intervention under trial conditions and subsequently to inform implementation strategies.

By utilising a theory-based mixed methods approach, we were able to identify key practices (such as the health professional knowing when to employ appropriate sections of the manual or implement practices from their wider professional repertoire) as well as configurations of best practice (such as the facilitator demonstrating the required exercise level and tailoring explanations). These demonstrate that persistence and change are dependent upon feedback between one instance or moment of enactment and another, and on patterns of mutual influence between co-existing practices [11].

Our findings indicate that the opportunities for achieving sustained behaviour change are contingent upon what the bundles of intervention practices demand of a person with heart failure and/or their caregiver and on previous life histories and resources accumulated along the way. This suggest that the provision of a simple blueprint or template for implementation into routine practice would be inappropriate considering the potential importance of tailoring the intervention to context and any change over time. Rather than a generalisable prescription our mixed methods approach provides critical transferable insights and greater explanatory power, allowing policy makers and practitioners (both lay and professional) to make informed decisions about future uptake of individual elements, bundles of elements, and the total REACH-HF package more generally [15, 43, 51, 52].

Our findings highlight the importance of one-to-one contact for participants. Thus, although such contact may be expensive, it may be critical to allow flexibility to ensure intervention impact across participants. Our findings also highlight how intervention delivery can vary across personal contexts, so emphasising the need to identify and train facilitators for best practice in intervention delivery before implementation. Our findings emphasise the value of both a Hybrid Type 1 trial design - synthesising randomised controlled trial and process evaluation data - prior to implementation. The integration of process and outcome data over time enabled us to make within-case and cross-case analyses. Such comparisons can highlight how well an intervention is designed to respond to a range of participant needs. This may direct intervention improvement prior to implementation (e.g. more focused facilitator training), as well as future implementation strategies (e.g. that align assessment of contextual factors, identification of patient and clinical goals, and the most appropriate intervention components).

Implications for research

This paper has a number of implications for future research. Commentators note that theoretical endeavours for implementation science often lack “how-to” support for carrying out implementation evaluation and interpretation [53]; can overlook the extent to which practitioners determine the processes and outcomes of implementation [54]; and the extent to which context is wider than the physical setting in which the implementation takes place [16, 17].

Social Practice Theory enabled us to identify that this complex behaviour change intervention was optimised when the competencies of the facilitator (e.g. rigorous assessment and an ability to challenge lay interpretations) and the intervention materials (e.g. the section of a manual or use of a patient tracker) were tailored to the goals and needs of the participant in ways which enabled them to transform the meaning (e.g. foster an appreciation of the value of exercise), thus enabling them to develop sustainable and flexible self-management practices. Sometimes, this was achieved despite a participant’s enduring problems; but not all facilitators were well acquainted with the intervention, or knowledgeable enough to deliver it well. If the problem is within the scope of the intervention (e.g. anxiety) then the tailoring is appropriate although fidelity may be compromised. However, if the problem is not within the scope of the intervention (e.g. material deprivation), then tailoring is very difficult. Our attention must now turn to how to deliberately steer or target practices and/or bundles of practices in the face of enduring health care and structural inequalities.

Hawe et al. contend that perhaps one of the reasons for interventions not working is that their components have been overly standardised, proposing that thinking about the real world context should become the ideal instead [14]. Our findings suggest that Social Practice Theory provides a framework for us to bring the social context back into process evaluations of complex interventions to inform implementation strategies in a meaningful way [14, 23].

We agree with Blue et al. [50] that complex interventions need to move beyond seeing people as static individual selves that require changing, to engaging people entwined in dynamic social arrangements who are defined and constituted through the practices that they adopt and enact [23]. We suggest that our use of Social Practice Theory to inform our analysis of multi-modal process and outcome data is a robust and dynamic theoretically informed methodological approach for evaluating intervention and implementation practices. We see this as having the potential to illuminate which configurations of elements need to be customised for every ‘user’ - such that any standardisation concerns the function of elements or bundles of elements that constitute interventions in a given context (e.g. unpacking Helen’s anxieties about death to enable her to participate in exercise), rather than the current focus on standardising the form of elements and their delivery (e.g. the deployment of the manual in Dorothy’s case without wider contextualisation) [55]. We suggest that this can help us moved toward the implementation of personalised interventions at scale [29].

Strengths and limitations

We have conducted the first mixed methods process evaluations of a comprehensive self-management intervention for people with heart failure and their carers. The use of a well specified sociological theory (Social Practice Theory) enabled the core elements of this complex intervention and their assemblages to be better articulated, explored and explained. This has enabled us to amplify the quantitative trial results and to use these insights to develop our subsequent implementation strategy.

We have acknowledged the limitations of this study elsewhere, in relation to other data we might have collected [32]. Our ongoing research and implementation activities aim to redress some of these limitations and build on our findings, with particular reference to the training and support of the facilitators [56, 57]. We have also noted the lack of diversity of our patient population, and it is also vital to address the particular challenges facing people from black and minority ethnic communities who are not well served by current interventions.

Conclusion

We conclude that Social Practice Theory has the potential to progress our understanding from a trial of a clinical intervention at T2 (translation to patients), to guide the development of an intervention strategy for implementation at T3 (translation to practice) via the identification of the most appropriate configurations of bundles of intervention elements for implementation in individual real-world clinical scenarios.

Availability of data and materials

Transcripts will not be shared in their entirety to protect the anonymity of participants and the facilitators delivering the intervention. However, requests for excerpts of the data will be considered on an individual basis. Please contact the corresponding author.

Abbreviations

ACE:

Angiotensin-converting-enzyme inhibitor

BMI:

Body Mass Index

COPD:

Chronic Obstructive Pulmonary Disease

CRT:

Cardiac resynchronisation therapy

HADS:

Hospital Anxiety and Depression Scale

ICD:

Implantable cardioverter defibrillator

LVEF:

Left ventricular ejection fraction

NYHA:

New York Heart Association (Classification)

MRC:

Medical Research Council

MLHFQ:

Minnesota Living with Heart Failure Questionnaire

NPT:

Normalisation Process Theory

REACH-HF:

Rehabilitation Enablement in Chronic Heart Failure

T2/3:

Time 2/Time 3

References

  1. Taylor RS, Hayward C, Eyre V. The clinical effectiveness and cost-effectiveness of the rehabilitation Enablement in chronic heart failure (REACH-HF) self-care rehabilitation intervention in heart failure patients and caregivers: rationale and protocol for a multicentre randomised controlled trial. BMJ Open. 2015;5:e009994.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. NICE. Chronic heart failure: management of chronic heart failure in adults in primary and secondary care: NICE ed. Chronic Heart Failure. National Institute for Health and Care Excellence. 2010.

  3. Greenhalgh T, A’Court C, Shaw S. Understanding heart failure; explaining telehealth – a hermeneutic systematic review. BMC Cardiovasc Dis. 2017;17(156):1–16.

    Google Scholar 

  4. Jonkman N, Westland H, Groenwold R, Ågren S, Anguita M, Blue L, Bruggink-André de la Porte P, Dewalt D, Hebert P, Heisler M, Jaarsma T, Kempen G, Leventhal M, Lok D, Mårtensson J, Muñiz J, Otsu H, Peters-Klimm F, Rich M, Riegel B, Strömberg A, Tsuyuki R, Trappenburg J, Schuurmans M, Hoes A. What are effective program characteristics of self-management interventions in patients with heart failure? An individual patient data meta-analysis. J Card Fail. 2016;22:861–71.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Clark A. What are the components of complex interventions in healthcare? Theorizing approaches to parts, powers and the whole intervention. Social Sci Med. 2013;93:185–93.

    Article  Google Scholar 

  6. Albano M, Jourdain P, de Andrade V, Domenke A, Desnos M, d’Ivernois JF. Therapeutic patient education in heart failure: do studies provide sufficient information about the educational programme? Arch Cardiovasc Dis. 2014;107:328–39.

    Article  PubMed  Google Scholar 

  7. Dalal H, Taylor R, Jolly K, Davis R, Doherty P, Miles J, van Lingen R, Warren C, Green C, Wingham J, Greaves C, Sadler S, Hilsdon M, Abraham C, Britten N, Frost J, Singh S, Hayward C, Eyre V, Paul K, Lang C, Smith K, on behalf of the REACH-HF Investigators. The effects and costs of home-based rehabilitation for heart failure with reduced ejection fraction: The REACH-HF multicentre randomized controlled trial. Eur J Prev Cardiol. 2019;26(3):262–72.

    Article  PubMed  Google Scholar 

  8. Taylor RS, Wingham J, Frost J, Britten N, Greaves C, Abraham C, Warren F, Jolly K, Miles J, Paul K, Doherty P, Singh S, Davies R, Green C, Eyres V, Hayward C, van Lingen R, Smith K, Lang C, Hillsdon M, Dalal H. On behalf of the REACH-HF research group. Facilitated self-care and rehabilitation for patients with reduced ejection fraction heart failure (REACH-HF trial): impact on caregiver outcomes. Submitted to European Journal of Cardiovascular Nursing.

  9. Westfall J, Mold J. Fagan L (2007) practice-based research: “Blue highways” on the NIHR roadmap. J Am Med Assoc. 2007;297(4):403–6.

    Article  CAS  Google Scholar 

  10. Dougherty D, Conway PH. The “3T’s” road map to transform US health care: the “how” of high-quality care. J Am Med Assoc. 2008;299(19):2319–21.

    Article  CAS  Google Scholar 

  11. Shove E, Pantzar M, Watson M. The dynamics of social practice: everyday life and how it changes. Los Angeles: Sage; 2012.

    Book  Google Scholar 

  12. Bauer M, Damschroder L, Hagedorn H, Smith J, Kilbourne A. An introduction to implementation science for the non-specialist. BMC Psychology. 2015;3:32.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Moore G, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, Baird J. Process evaluation of complex interventions: Medical Research Council guidance. London: MRC Population Health Science Research Network; 2014.

    Google Scholar 

  14. Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randaomised controlled trail be? Brit Med J. 2004;328:1561–3.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Oakley A, Syrange V, Bonell C, Allen E, Stephenson J, RIPPLE study team. Process evaluation in randomised controlled trials of complex interventions. Brit Med J. 2006;332:413–6.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Hoddinot P, Britten J, Pill R. Why do interventions work in some places and not others: A breast feeding support group trial. Social Sci Med. 2010;70:769–78.

    Article  Google Scholar 

  17. Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Grimshaw J, Presseau J, Tetroe J, Eccles M, Franci J, Godin G, Graham D, Hux J, Johnston M, Legare F, Lemyre L, Robinson N, Zwarenstein M. Looking inside the black box: results of a theory-based process evaluation exploring the results of a randomized controlled trial of printed educational messages to increase primary care physicians’ diabetic retinopathy referrals. Implement Sci. 2014;9:86.

    Article  PubMed  PubMed Central  Google Scholar 

  19. May CR, Cummings A, Girling M, Bracher M, Mair FS, May CM, Murray E, Myall M, Rapley T, Finch T. Using normalization process theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review. Implement Sci. 2018;13(80).

  20. Whitty C. What makes an academic paper useful for health policy? BMC Med. 2015;13:301.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Clarke DJ, Godfrey M, Hawkins R, Sadler E, Harding G, Forster A, McKevitt C, Dickerson J, Farrin A. Implementing a training intervention to support caregivers after stroke: a process evaluation examining the initiation and embedding of programme change. Implement Sci. 2013;8(1):96.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Braithwaite J, Churruca K, Ellis L, Herkles J. When complexity science meets implementation science: atheortical and empirical analysis of systems change. BMC Med. 2018;16(63).

  23. Craig P, Di Ruggiero E, Frohlich KL, Mykhalovskiy E, White M, On behalf of the Canadian Institutes of Health Research (CIHR)–National Institute for Health Research (NIHR) context guidance authors group. Taking account of context in population health intervention research: guidance for producers, users and funders of research. Southampton: NIHR Evaluation, Trials and Studies Coordinating Centre; 2018.

    Book  Google Scholar 

  24. Holman D, Lynch R, Reeves A. How do health behaviour interventions take account of social context? A Literature trend and co-citation analysis. Health. 2018;22(4):389–410.

    Article  PubMed  Google Scholar 

  25. Hargreaves T. Practice-ing behaviour change: applying social practive theory to pro-environmental behaviour change. J Consum Cult. 2011;11(79).

  26. Shove E, Trentmann F. Infrastructure in practice: the dynamics of demand in networked societies. Oxon: Routledge; 2018.

    Book  Google Scholar 

  27. Hampton S. Policy implementation as practice? Using social practice theory to examine multi-level governance efforts to decarbonise transport in the United Kingdom. Energy Res Soc Sci. 2018;38:41–52.

    Article  Google Scholar 

  28. Bunn C, Wyke S, Gray C, Maclean A, Hunt K. ‘Coz football is what we all have’: masculinities, practice, performance and effervesce in gender-sensitized weight-loss and healthy living programme for men. Sociology Health Illness. 2016;38(5):812–28.

    Article  Google Scholar 

  29. Shaw J, Shaw S, Wherton J, Hughes G, Greenhalgh T. Studying scale-up and spread as social practice: Theoretical introduction and empirical case study. J Med Internet Res. 2017;19(7):e244.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Lothian Health Board. The heart manual (post MI edition). Edinburgh: Lothian Health Board; 2011.

    Google Scholar 

  31. Greaves CJ, Wingham J, Deighan C, et al. Optimising self-care support for people with heart failure and their caregivers: development of the rehabilitation Enablement in chronic heart failure (REACH-HF) intervention using intervention mapping. Pilot Feasibility Stud. 2016;2:37.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Frost J, Wingham J, Britten N, Greaves C, Abraham C, Warren F, Jolly K, Doherty P, Miles J, Singh S, Paul K, Taylor R, Dalal H. Home-based rehabilitation for heart failure with reduced ejection fraction: process evaluation of the REACH-HF multicentre randomised controlled trial. BMJ Open. 2019;9:e026039.

    Article  PubMed  PubMed Central  Google Scholar 

  33. American Thoracic Society. Minnesota living with heart failure questionnaire, New York: American Thoracic Society, 2004. . http://qol.thoracic.org/sections/instruments/ko/pages/mlwhfq.html.

  34. EuroQoL G. EuroQol – a new facility for the measurement of health-related quality of life. Health Policy. 1990;16:199–208 [PubMed] [Google Scholar].

    Article  Google Scholar 

  35. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67:361–70.

    Article  CAS  PubMed  Google Scholar 

  36. Pulz C, Diniz RV, Alves AN, et al. Incremental shuttle and six-minute walking tests in the assessment of functional capacity in chronic heart failure. Can J Cardiol. 2008;24:131–5.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Van den Berg-Emons HJ, Bussmann JB, Balk AH, et al. Validity of ambulatory accelerometry to quantify physical activity in heart failure. Scand J Rehabil Med. 2000;32:187–92.

    Article  PubMed  Google Scholar 

  38. Oldridge N, Hofer S, McGee H, et al. The HeartQoL: part II. Validation of a new core health-related quality of life questionnaire for patients with ischemic heart disease. Eur J Prev Cardiol. 2014;21:98–106.

    Article  PubMed  Google Scholar 

  39. Riegel B, Lee CS, Dickson VV, et al. An update on the self-care of heart failure index. J Cardiovasc Nurs. 2009;24:485–97.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Criteria Committee of the New York Heart Association. Nomenclature and criteria for diagnosis of diseases of the heart and great vessels. 9th ed. Boston: Little, Brown & Co; 1994.

    Google Scholar 

  41. StataCorp. Stata statistical software: release 14. College station: StataCorp LP; 2015.

    Google Scholar 

  42. QSR International Pty Ltd. NVivo qualitative data analysis software; version 10; 2012.

    Google Scholar 

  43. Bazeley P. Integrating analysis in mixed methods research. London; Sage. 2018.

  44. Miles M, Huberman A, Saldana J. Qualitative data analysis: a methods sourcebook. Los Angeles: Sage; 2014.

    Google Scholar 

  45. Saldana J. Longitudinal qualitative research: Analysing change through time. California: AltaMira; 2003.

    Google Scholar 

  46. Hoffman T, Glasziou P, Boutron I, Milne R, Perera R, Moher D, Altman D, Barbur V, Macdonald H, Johnston M, Dixon-Woods M, McCulloch P, Wyatt J, Chan A, Michie S. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. Br Med J. 2014;348:g1687.

    Article  Google Scholar 

  47. Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf. 2015;24:228–38.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Dalal H, Doherty P, Taylor R. Cardiac rehabilitation. Br Med J. 2015;351:h5000.

    Article  CAS  Google Scholar 

  49. Gallacher K, May C, Montori V, Mair F. Understanding Patients’ Experiences of Treatment Burden in Chronic Heart Failure Using Normalization Process Theory. Ann Fam Med. 2011;9(3):235–43.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Blue S, Shove E, Carmona C, Kelly M. Theories of practice and public health: understanding (un)healthy practices. Critical Public Health. 2016;26(1):36–50.

  51. Goodyear L, Barela E, Jewiss J, Usinger J. Qualitative inquiry in evaluation: from theory to practice (research methods for social sciences). California: Jossey- Bass; 2014.

    Google Scholar 

  52. O’Cathain A. A practical guide to using qualitative research with randomised controlled trails. Oxford: Oxford University press; 2018.

    Book  Google Scholar 

  53. Nilsen P. Making sense of implementation theories, models and framework. Implementation Science. 2015;10:53.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2016;11:33.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Haynes A, Brennan S, Redman S, Williamson A, Gallego G, Butow P, CIPHER team. Figuring out fidelity: a worked example of the methods used to identify, critique and revise the essential elements of a contextualised intervention in health policy. Implement Sci. 2016;11(23):1–18.

    Google Scholar 

  56. Lowery J, Hopp F, Subramania U, Witalia W, Welsh D, Larkin A, Stemmer K, Zak C, Vaitkevicius P. Evaluation of a nurse practitioner disease management model for chronic heart failure: a multi-site implementation study. Congestive Heart Failure. 2012;18:64–71.

    Article  PubMed  Google Scholar 

  57. Macarthy M, Dickinson V, Katz S, Sciacca K. Process evaluation of an exercise counselling intervention using motivational interviewing. Appl Nurs Res. 2015;28:156–62.

    Article  Google Scholar 

Download references

Acknowledgements

We thank the patients and caregivers who took part in the REACH-HF randomised controlled trial and process evaluation. We thank Kevin Paul for his contribution as patient and public involvement representative. We acknowledge the contribution of the Heart Manual Office at NHS Lothian who contributed to the design of the intervention and facilitator training and the Peninsula Clinical trials Unit for study management. Jo Day provided insightful comments on this manuscript. We thank the reviewers for their comments and suggestions.

The views expressed in this publication are those of the authors and not necessarily of the NHS, NIHR, or UK Department of Health and Social care. We would like to thank all facilitators and clinicians involved in the study and acknowledge the vital contributions of study researchers and administrators in Birmingham, Cornwall, Gwent, the Heart Manual Office (Dr Carolyn Deighan and Jenny Edwards for the facilitator training), the Peninsula Clinical Trials Unit, the Royal Cornwall Hospitals Trust (Research, Development and Innovation and Clinical Chemistry departments), the Programme Steering Committee (Martin Cowie, Graham Dunn, Suzanna Hardman, Roger Boyle, and Liz Clark), the Data Monitoring Committee (Alan Montgomery, Ann Dorthe-Zwisler, and Gill Furze) and independent adjudicators (Iain Squire, Sern Lim, and Paco Leyva).

Reporting standards CONSORT-SPI.

Funding

REACH-HF was funded by a Programme Grant for Applied Research (RP-PG-1210-12004) awarded by the National Institute for Health Research (NIHR). The NIHR had no contribution to the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

All authors have read and approved the manuscript. JF: lead for mixed methods data analysis, drafting of paper. JW: Study design, part of the intervention development team, selection of participants, qualitative data collection and analysis, analysis of all facilitator recordings, facilitated patient and public involvement, design and application /analysis of the intervention fidelity checklist, critical review of the paper. NB: Study design, supervision of data collection and analysis, analysis of facilitation recordings, critical review of draft paper. CG: Study design, lead for intervention development team, design and application /analysis of the intervention fidelity checklist, contribution to data interpretation and critical review of the paper. CA: Co-wrote the original grant, contributed to study design and data interpretation and critically reviewed the paper. FW: Quantitative analysis of the REACH-HF trial and critical review of the paper. HD: Co-chief Investigator of the REACH-HF trial and commented on drafts of the paper. RT: Co-chief Investigator of the REACH-HF trial and commented on drafts of the paper.

Corresponding author

Correspondence to Julia Frost.

Ethics declarations

Ethics approval and consent to participate

Ethical approval was granted by the North West – Lancaster Research Ethics Committee (ref 14/NW/1351). Informed written consent was obtained from all participants. Trial registration number ISRCTN86234930.

Consent for publication

Participants provided written consented for anonymised data to be used in publications, and names in this paper are pseudonyms.

Competing interests

None declared.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Frost, J., Wingham, J., Britten, N. et al. The value of social practice theory for implementation science: learning from a theory-based mixed methods process evaluation of a randomised controlled trial. BMC Med Res Methodol 20, 181 (2020). https://doi.org/10.1186/s12874-020-01060-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12874-020-01060-5

Keywords