Application of the critical incident technique in refining a realist initial programme theory

Abstract

Background

As realist methodology is still evolving, there is a paucity of guidance on how to conduct theory driven interviews. Realist researchers can therefore struggle to collect interview data that can make a meaningful contribution to refining their initial programme theory. Collecting data to inform realist Inital Programme Theories (IPTs) in healthcare contexts is further compounded due to the healthcare workers’ busy work schedules.

In this case study of team interventions in acute hospital contexts, we explore the benefits of using the Critical Incident Technique (CIT) in order to build and refine an initial programme theory. We contend that use of the CIT helps to draw on more specific experiences of “Key Informants” and therefore elicits richer and more relevant data for realist enquiry.

Methods

The five steps of the CIT were mapped against realist methods guidance and adapted into an interview framework. Specifications to identify an incident as “critical” were agreed. Probes were embedded in the interview framework to confirm, refine and/or refute previous theories.

Seventeen participants were interviewed and recordings were transcribed and imported for analysis into NVivo software. Using RAMESES guidelines, Context-Mechanism-Outcomes configurations were extrapolated from a total of 31 incidents.

Results

We found that the CIT facilitated construction of an interview format that allowed participants to reflect on specific experiences of interest. We demonstrate how the CIT strengthened initial programme theory development as it facilitated the reporting of the specifics of team interventions and the contexts and mechanisms characteristic of those experiences. As new data emerged, it was possible to evolve previous theories synthesised from the literature as well as to explore new theories.

Conclusions

Utilising a CIT framework paid dividends in terms of the relevance and usefulness of the data for refining the initial programme theory. Adapting the CIT questioning technique helped to focus the participants on the specifics relating to an incident allowing the interviewers to concentrate on probes to explore theories during the interview process. The CIT interview format therefore achieved its purpose and can be adapted for use within realist methodology.

Background

As realist evaluation is still evolving, there is a paucity of guidance to support the appropriate use and selection of methods within a realist approach. Whilst researchers are encouraged to use an open, flexible and iterative approach to programme theory development and refinement [1], there is little published on the specifics of how to do this. Realist methods are theory driven and in keeping with an iterative and interpretative process. They take into account a broad range of perspectives and seek to deepen understanding of ‘what works, for whom, in what conditions, why, to what extent and how?’ [2]. Realist methods have therefore been increasingly commissioned by health policy makers to inform complex health interventions [3,4,5].

See Table 1 for a definition of Context, Mechanism, Outcome and Configuration.

Table 1 Definition: Context, Mechanism, Outcome Configuration (CMOC)

The aim of the realist researcher is to explore theories or hypotheses with regard to how and why programmes or interventions work or do not work by engaging in a narrative about the theory with those who have specialist knowledge and are therefore considered Key Informants- hereafter referred to as participants. This construction of a narrative is usually achieved via an interview process which is designed around the participants’ experience and reasoning with regard to the programme or intervention being evaluated [6]. In order to “theorise the interview”, Pawson advocates for use of Teacher-Learner style interviews where the realist interviewer is urged to take an active role in directing the line of questioning whilst ensuring that the subject matter under evaluation - the programme theory - remains the focus of the interview [7]. For the realist researcher, this can be a demanding task and there is limited guidance in the literature. Manzano’s methods [8] of theory gleaning, refining and consolidation have important application to theory testing in realist evaluation, however collecting data for the purpose of building and refining an initial programme theory, is less well explored. In this paper we explore the application of Flanagan’s Critical Incident Technique (CIT) [9] as a technique to elicit the nuance and richness required in developing initial programme theory.

Flanagan describes CIT as “a flexible set of principles that must be modified and adapted to meet the specific situation at hand” [9] p. 335. Specifically,

“CIT research takes place in a natural setting; the researcher is the key instrument of data collection; data are collected as words through interviewing, participant observation, and/or qualitative open-ended questions; data analysis is done inductively; and the focus is on participants’ perspectives” [10] p 16.

The CIT includes five steps in the form of procedures: determination of the general aim of the activity; development of plans and specifications for collecting factual incidents regarding the activity; collection of the data; analysis of the data and interpretation and reporting of the statement of the requirements of the activity. It is apparent from its use in other healthcare studies that these steps can be tailored to specific situations [11, 12]. We sought to explore whether it was possible to maintain this procedural integrity within the more flexible, open and iterative processes required for realist methods.

This technique has already been adapted to explore contextual detail in incidents deemed of critical importance to individuals [13, 14]. As Creswell has already placed CIT in a qualitative framework [15], it has potential to extend its application to realist methods. Woolsey [16] has previously recognised its usefulness in the early stages of research as a foundational exploratory tool and for its role in building theories. We hypothesised therefore that this method could also have potential in this case study. As the CIT allows for in-depth exploration of the antecedents and consequences to a specific incident, it aligns closely to the configuration of context, mechanism and outcomes within the realist evaluation.

Realist methodology calls for interpretive analysis to gain deeper insights on how and why contexts generate outcomes. It is underpinned by a critical realism philosophy [17]. Research in this post-positivist mode requires taking a distanced view [10] and it is accepted that we can identify what we do not directly observe using the practical and theoretical processes of the social sciences.

Chell argues that although emanating from a positivist paradigm, CIT can also be used ‘within an interpretive or phenomenological paradigm’ [18] p 51. Given this, we seek to explore the feasibility and usefulness of quality of data obtained using the CIT. To our knowledge, the application of CIT in realist methods has not previously been examined. This article therefore:

• 1. Explores use of CIT within a realist methods framework

• 2. Determines if CIT has potential to be used in refining an initial programme theory

• 3. Demonstrates application of the CIT via this case study of team interventions in acute hospital contexts.

Contributions to the literature

For the realist researcher, there is a paucity of data on how to conduct interviews for the purpose of refining an initial programme theory.

Although Flanagan’s CIT has been adapted for use in several domains, to our knowledge, its application to realist methods has not been explored.

• Findings contribute to the literature by demonstrating that it is possible to maintain the integrity of the five procedures described in CIT whilst placing the technique into a realist framework. CIT offers researchers a structured, adaptable and practical guide on how realist researchers may apply CIT in refining an initial programme theory.

• The CIT encourages the interviewee to focus on a specific event or incident and in doing so reduces any tendency to provide generic or “woolly” statements that tend to be of limited benefit in developing theory.

• The CIT can enhance initial programme theory development as it facilitates the elicitation of the nuance and specifics of an individual’s experiences and the contexts and mechanisms characteristic of those experiences through the in-depth reflection on those experiences.

Use of CIT allowed participants to recall detail of interventions in an efficient manner. The majority of data retrieved had relevance for initial programme theory building giving a concomitant efficiency to the research team during the data analysis phase.

Methods

We employed a case study for the purposes of illustrating how the critical incident technique may be applied in realist methods. This study examined the contextual conditions for team interventions in acute hospital contexts and more specifically, the enablers and barriers to team intervention success. It built on previous work which involved a systematic search of the literature using realist synthesis [19].

For the purpose of this research, a multi -disciplinary team intervention was defined as:

An intervention where a team of two or more disciplines is trying to improve how the team delivers patient care- for example: quality improvement, service improvement or change initiatives; process re-design or team training events.

These interventions were considered as complex social interventions [2] and realist evaluation was therefore considered an appropriate methodology having already been used in similar studies [12, 20, 21]. The process commenced with a systematic search of the literature which was driven by the researcher’s own experiential knowledge and assumptions [19]. Five plausible hypotheses were extrapolated using realist synthesis and are presented in the form of Context, Mechanism, and Outcome Configurations (CMOCs). Please refer to Table 2 [19].

Table 2 Plausible Hypotheses

Using these hypotheses as a foundation for the programme theory, during interviews with hospital staff who had been involved in team interventions, the authors sought to explore the conditions in which these interventions were introduced (Contexts- C); how the resources on offer in these particular contexts permeated into the reasoning of those involved in the team intervention (Mechanisms-M) and the intended and un-intended consequences of the intervention (Outcomes- O).

Please refer to Fig. 1.

Fig. 1

Overview of the initial programme theory development process

Difficulties for the realist researcher in conducting research have already been cited [20]. In this case study, the busyness of the acute hospital had potential to further impact the fieldwork process. Scheduling interviews during daily routines meant hospital staff had to consciously shift their mind-set from clinical or operational activity to the more reflective mode required for research interviews.

Prior to consideration of the CIT, trial interviews using a semi-structured format had been piloted with two purposively sampled hospital staff -one female hospital operations manager and one female hospital therapist both of whom had led on team interventions. These semi- structured interviews included open ended questions for example:

“Tell me about an intervention that you have been involved in” and “How did the team operate?”

The research team reviewed the data that emerged from these interviews and agreed that significant portions of the narrative consisted of tangential generalities about teams rather than specific information related to the intervention. Participants found it difficult to construe the intervention, speaking about the team in an abstract and sometimes detached way, as evidenced in one response:

“I am not even sure of who was involved in that one …some members were only pulled in when they were needed to solve that piece of the puzzle…” [Semi-structured interview 1]

Participants demonstrated poor recall of specifics about the team or the context in which they were operating focussing instead on the process, problem or issue for which the intervention was designed. In addition, significant tangential information was collected relating to individual work patterns, relationships and practices which had limited relevance for programme theory refinement:

“Some days are cruel, you know, especially when I am in two different places…I came in this morning at half seven and I haven’t had my lunch yet…I’ve a clinic after this I need to get to, if {Name} is on, I am snookered …” [Semi Structured Interview 2]

Use of this semi-structured approach had not extracted the necessary detail to meaningfully contribute to programme theory development (i.e., there were insufficient data relevant to context and mechanisms) and the research team agreed that a different format was required.

Researchers agreed that it would be necessary to elicit participant experiences before, during, and after a team intervention/programme was implemented in order to explore contextual conditions for programme theory building [8]. In order to increase the quality and value of the data collected, the researchers therefore considered use of Flanagan’s Critical Incident Technique (CIT) [9] framework.

To ensure rigour in this approach, the research team first mapped and compared the five CIT procedures against the characteristics and features of realist methods (Tables 3 and 4).

Table 3 Mapping Critical Incident Technique against Realist Methodology-General principles

Table 4 Critical Incident Technique mapped against Realist Methodology for the purpose of the case study

Recruitment

Fifteen participants who had been involved in team interventions were purposively sampled initally by either the Chief Operating Officer or General Manager of each participating organisation to reflect a range of disciplines; gender balance and healthcare experience across four acute hospitals in one Irish Hospital Group. Demographic information on participants is presented in Table 5 (Subsequently two additional participants were interviewed to explore an emerging theory further: KI16 & KI17).

Table 5 Key Informant Descriptors

Participants were invited to participate in the process by e-mail correspondence 1 week in advance of the interviews. The e-mail correspondence included an information sheet and consent form. Participants were advised that participation was voluntary and that their responses would be confidential.

Prior to the interviews being conducted, the researchers considered the application of each of the CIT procedures to the format of the interview.

CIT procedure 1- development of plans and specifications

The research team clarified the purpose of the critical incident interview technique (as relevant to the case study described) and agreed how to unpick the relevance of the team intervention that the participant was describing. The objective of the CIT interviews was to obtain information relevant to: team descriptors; contextual conditions (C); the objective of the intervention; outcomes (O). Probes were included with regard to how and why an intervention worked in order to elicit the mechanisms enacted (M). The interview guide was designed to give specific attention to the five CMOCs already synthesised from the literature whilst also allowing other contextual enablers and barriers to emerge and the subsequent mechanisms and outcomes generated by those conditions.

CIT procedure 2-determination of the general aim of the activity

The research team agreed that incidents were “critical” if participants deemed them to be significant in terms of their experience and if they could be relied on as relatively accurate accounts of specific events. A “critical incident” for the purpose of this study was defined as:

“a team intervention recalled by the participant as either a significant positive or negative experience that meets research criteria in terms of being a multi-disciplinary team intervention”.

The team considered that incidents meeting this definition were more likely to meaningfully contribute to building the IPT.

As per the CIT, interviews commenced with an introductory statement to advise participants of the purpose of the exercise (Please refer to Appendix 1. p.1 for complete protocol). Following some background questions regarding professional roles and experience, participants were asked to recall a critical incident as follows:

• 1. Can you think of a significant event/situation/time that you were particularly proud of working on a team intervention or initiative to improve patient care?

• 2. In a few words can you tell me what was the primary aim of the initiative was?

If recalled incidents were not deemed to meet the research criteria, participants were re-directed for the purpose of the exercise, for example:

“That’s a really nice example of an intervention introduced with your own professional colleagues, I am going to ask you to think again … .this time if you can think of an intervention where there were a number of disciplines involved, that would be great.”

Following an initial question as to why they selected this experience, they were then asked a series of questions with probes embedded to elicit more factual data specific to the intervention experience. This process was followed by asking participants to recall a significant event /situation/time that they were particularly proud of, and subsequently, one there were not so proud of (See Appendix1Interview).

CIT procedure 3 -collecting the data

Tests of critical incident interview format

As with the semi-structured format, the primary researcher (UC) pilot tested the new critical incident interview format on two purposefully chosen hospital staff who had been involved in leading team interventions. Following this, minor changes were made. For example, additional prompts were included to ensure interviewers probed for detail with regard to the existing CMOCs by including prompts for these in a different colour. As per the CIT, it was agreed that interviewer remarks should be neutral and permissive [22] and should demonstrate that the interviewee was the expert. However, if specifics were not emerging, clarifications could be sought for example:

“So what you are saying is… .?” or “Can you give me more detail on that?”

Similarly, if information was ambiguous, interviewers could say …

“I am not sure I understood that point, am I correct in saying … .?”

Following the critical incident interview trials, additional probes for data to confirm, refute or refine theories that had worked well were considered and researchers agreed on the final format for the interview (see Appendix 1 supplementary materials for final interview guide).

Interviews

Interviews were conducted over a period from May to September, 2018 by two members of the research team (UC & ADB). Interviews were audio-recorded with participants’ consent and transcribed verbatim. One participant did not consent to audio recording and therefore notes were taken by the interviewer during the interview process.

CIT procedure 4 Analysing the data

Interviews were transcribed, anonymised and imported into NVivo software for the purpose of storage, analysis and interrogation of the data [23].. NVivo memo and annotation functions were used to document thought processes and decision making thus allowing the iterative process of theory building to be captured. This helped to ensure transparency [24,25,26] thus adhering to RAMESES II reporting standards as well as allowing for the requisite objectivity demanded by the CIT.

A critical incident was the unit of analysis (i.e., each incident described by the participant was one data unit, N = 29 incidents, as one participant could not recall a negative experience. Data were subsequently analysed in three phases summarised in Table 6 and detailed thereafter under phase 1, phase 2 and phase 3 headings. Please refer to Table 6.

Table 6 Summary of Data Analysis- Flow chart

As recommended in the Rameses II Quality Standards for Realist Evaluation [27], a retroductive approach was adopted for analysis of the data. Retroduction refers to the movement between inductive and deductive processes to explain how and why things work the way they do. Realist researchers seek to explain the hidden causal powers of an intervention in the context in which it is applied [28]. Realist principles of retroduction stress the need for iteration within theory refinement [24]. Going back and forth using both a deductive and an inductive lens enables comparison across units of analysis. The retroductive process used was consistent with the inductive lens required for CIT and also allowed existing CMOCs to be tested using a deductive lens.

Using deductive logic, data were first analysed against the five existing theories (CMOCs) that had been extrapolated from the literature in the form of plausible hypotheses to see whether associations matched the expectations. Consistent with the CIT approach, an inductive lens was also applied to the data as narratives were reviewed to extrapolate evidence that suggested the emergence of new theories. In keeping with the iterative process, the research team and the realist researcher methodology support group* were consulted to provide advice and feedback at each phase of the evaluation process. Use of NVivo software helped to support and document analytical steps and decision-making in line with a retroductive approach to analysis [24].

Please refer to Table 7for overview of consultations sessions.

Table 7 Overview of Consultation sessions

* An interdisciplinary group of researchers and academics with a specific interest in, and experience in applying, in realist methods.

Phase 1 data analysis: scanning transcripts- induction and deduction

Using the five CMOCs that had emerged from the literature as parent nodes, phase 1 analysis involved an initial scanning of the transcripts. Pieces of narrative were coded according to parent nodes “team descriptors” and “CMOCs 1–5” Nodes were also created to explore new potential CMOCs.

A piece of narrative was annotated if it was judged to be a relevant observation relating to the theory; for example, if it demonstrated a moderating function or appeared to refute, support or confirm prior findings. Where there was evidence of a new contextual enabler or barrier emerging, a memo was written to document how and why it was perceived to be so and to record the rationale for decisions made.

In parallel to the coding process in NVivo, a programme theory template was also developed in hard copy with each phase of the analysis colour coded to demonstrate the evolving theories and/or new emerging theories (Available on request from primary researcher UC). This was done because the primary researcher (UC) had a personal preference for reviewing the theories with the research team in hard copy as the evolution of theory refinement was considered easier to view and be interrogated at a glance.

Phase 2 data analysis: building and refining theories- Retroduction

Data that were coded under the 5 original CMOCs were re-analysed and re-coded against 3 child nodes: support/ refute/ refine to allow for transparency of the process.

Narratives coded under “New CMOC” in Phase 1 were re-analysed under eight emerging theories. How and why they resulted in an intended or un-intended outcome was queried. During this process, evidence to support, refute or refine the enabling condition was first extrapolated. For transparency of the process, each of the eight emerging CMOCs were used as parent nodes and narrative was coded if there was evidence specific to Context, Mechanism and Outcome.

Phase 3 analysis

As part of the iterative process of data analysis, additional notes were made if there was evidence of moderating influences, rival mechanisms and inter-dependencies. Where refinement of the theory appeared to be indicated, an annotation was made in NVivo as to how and why the judgment for same was made. Where a judgment call could not be made by the primary researcher, a memo link was created for discussion with co- authors and the realist support group.

Both groups suggested further exploration specific to one possible new theory - “in the moment learning”. Two additional interviews were therefore undertaken in December 2018 with purposively sampled participants who had specific expertise in delivering team interventions using event simulation as a team training intervention. Two additional positive experiences of team interventions (N = 2 incidents) were analysed following the same three phase analysis (Total N = 31 incidents). Following this iterative process of data analysis, the research team agreed and finalised the initial programme theory.

CIT procedure 5- interpreting and reporting the data

As per the CIT, it is “imperative” that interpretation and reporting of data “is objective” [22]. For this purpose, RAMESES II for realist evaluation were followed [28]. In addition, the consolidated criteria for reporting qualitative studies (COREQ) was adhered to [29] . In order to understand and agree the underpinning cause of the outcomes observed, data were presented to the research team and the realist support group on two separate occasions so that the chains of inferences (CMOCs) made by the primary researcher could be challenged. This helped to maintain objectivity and rigour in the process of developing insights.

Results

Findings relating to how the application of CIT complemented realist methods and its potential in refining initial programme theory are presented below. In addition, the benefit of using CIT within this case study is outlined.

How the application of CIT complemented realist methods

CIT procedures 1 and 2 helped to focus the researchers on the purpose of the interview and that questions must relate to teamwork and building of programme theory (See Appendix 1 p1).

The mapping exercise comparing CIT and realist paid dividends in terms of the quality, relevance and usefulness of the data collected during the critical incident interviews. In contrast to the generalities elicited during semi-structured interviews, participants appeared to have better recall and to connect immediately with the specifics of the team when describing specific interventions:

[KI 2] Pointing to chairs around the room

“The facilities manager would have been involved in it at the time, the healthcare records manager would have been involved in it at the time, I would have been involved in it from a quality and patient safety point of view, the divisional nurse manager was involved …I suppose in this room where we’re sitting now most of the meetings took place”.

In addition, drawing on specific interventions appeared to facilitate an emotional connection with the incident. Participants frequently re-constructed scenarios:

[KI 5] “Because we stood up in front of them and presented that we had taken 1600 patients off their waiting lists and of those 1600 patients we had asked them to see I think 112 so we were able to say look of these 1600 patients that were on your waiting lists, … … ... That’s what this programme does for your service… ..and that was the sea change point because the data was so strong and the relationship changed, literally changed almost overnight”.

Participants often unconsciously referred to how and why things happened the way they did, linking context and outcomes unpacking the mechanisms that were enacted as illustrated below (emphasis and (M) mechanism indication added):

[KI 4] “It is important that they are aware of why we want to do something … and they feel when you engage with them as well that you are appreciating that you know that the importance of their role in the hospital and it gives them a sense I suppose of value (M) … . that they are valuable resource but that they are key, they’re key support in the hospital in terms of patient care”.

If detail was unclear or did not emerge, interviewers were instructed to seek clarifications and or to elicit detail regarding team behaviours, actions and observations using specific questions:

[Q9] Has anything changed as a result of this initiative? If so, how? Probe: What was the outcome for patient care in this event… ..The outcome for the team in this event… ? How did you react to this? How did you feel as a result? How did the team react to this? / How did the team feel as a result?

And these often stimulated considered responses:

[KI 15] “Well I suppose we would have had that level of trust beforehand and it was just a reinforcement of that level of trust (C) … , I think if we hadn’t had that level of trust beforehand and mutual respect beforehand it wouldn’t have happened in [Name of hospital] in the first place (O) … . but it obviously makes you feel more valued in terms of as a peer (M) in terms of your skill levels but that’s something that you build up over time”

In this way, participants were led by the interviewer via open questions as per below.

Q 7 How did the team operate?

Embedded prompts then suggested interviewers might have the opportunity to explore two plausible hypotheses depending on how the participant responded to this question. This helped to ensure the interviewer did not limit the participant’s response.

Q 7 Probe for PH 1 (Interdisciplinary approach and flattened hierarchy) and PH2 (Effective communication)

These prompts helped to marry the purpose of the interview which was to refine the initial programme theory (realist methods) with participant responses relating to events and facts, i.e., team behaviours and actions associated with the events (CIT).

Potential of CIT in refining initial programme theory

This technique was successfully used to refine the initial programme theory. Seven patterns of occurrence of CMOCs were elicited from participant narratives. Two consequential CMOCs in the form of “ripple theories” (i.e., where the outcome of one CMOC became a contextual condition to generate a subsequent CMOC) also occurred with regularity. These chains of inference (IPT) are presented in the form of nine “If-then” statements below – See Table 8 below.

Table 8 Initial Programme Theory

New enabling or disabling conditions were also considered during the first phase of data analysis (CIT Procedure 4) and coded as “possible” theories relating to: competing demands; supportive function of the team; in the moment learning or source of drive for the intervention. However, researchers did not find a regular pattern of occurrence across the participant narratives to support these chains of inference. During the iterative processes of programme theory development in Phase 2 and 3 (CIT Procedure 4), these contextual variables were either discounted; judged to be intrinsic to another CMOC or judged to be a moderating function to one of the 7 confirmed theories or 2 ripple theories. Participants did not refute any theories synthesised from the literature thus perhaps strengthening them further.

How CIT benefitted initial programme theory refinement in the study of team interventions in acute hospital contexts

New information that had not been synthesised from the literature emerged from participant narratives. By specifically focussing on the antecedents to interventions and seeking the detail on how and why they impacted outcomes, prior working relationships (CMOC 6) emerged as a key contextual enabler across 6 incidents. By specifically asking for examples of negative experiences Inter-professional tensions emerged as a barrier to intervention success across seven incidents (CMOC 7). For some participants, this was conceptualised as specific individuals having the power to de-rail an intervention process (KI5, KI7, KI11). On more objective analysis however (CIT procedure 5) and discussion of the detail of narratives with the realist support group, it was agreed that the barrier was more likely to result from the broader issue of inter-professional tensions and rivalries and thus CMOC 7 and CMOC 7a chains of inference were made.

The flexibility of the CIT meant that interviewers could explore existing theories in more detail thus allowing refinement of theory to occur. For example, in Question 5 relating to the structure and function of the team, interviewers were encouraged to probe in relation to physician engagement PH 5. From this, a pattern emerged across 6 narratives relating to the mechanism enacted. PH 5 therefore required refinement. For participants it was the enactment of a perception of power and influence by physician engagement that results in legitimacy of the intervention and yielded the better outcome for the intervention.

[KI 4]: Well I mean as I say when you look at the patient pathways to see that there’s senior clinicians involved (C) in the process means that you know that the rest of the team see that it’s taken seriously (M) I think you know what I mean. That it’s given that level of importance and that they know that by having these people engaging they’re listening and they will address the issues for them and working with the patient and for the patient’s journey as well it will improve by having those (O).

Where inter-dependencies between theories seemed to be suggested, the flexibility of the CIT allowed interviewers to explore how and why in subsequent interviews. For example, use of effective communication and Smart, Measurable, Achievable, Realisitc, Time-bound goals (CMOC 3); and physician engagement and broad team composition (CMOC 5) both appear to give a sense of credibility and are thus related to CMOC 4. Upon detailed exploration with participants, these conditions appeared to be associated with a better chance of success of the intervention and it is, this association with success that causes high self and team satisfaction; increased team skills, increased team efficacy leading to a successful outcome and positive team reputation and this has a further escalating effect:

… empowering motivating and incentivising staff resulting in externally perceived credibility in the intervention and subsequent buy in with increased likelihood of further engagement and spread of the intervention and/or future team interventions Ripple CMOC 4.

In the absence of the specific detail that CIT elicits, this important information might not have emerged.

Discussion

Engaging in a social construction of a narrative about team interventions in acute hospital contexts is challenging and the importance of both the interview process and the interviewer as the ‘prime research instrument’ cannot be under-estimated [7, 8]. As opposed to a positivist approach, where the assumption is that “the researcher is independent of and neither affects nor is affected by the subject of the research” [30] p33, during the realist interview process, the interviewer is considered integral in the development of the theory [1, 8]. In the early stages of programme theory development, the realist interviewer needs to be mindful however that they do not lead the Key Informants who are considered the experts in the subject matter and striking this balance can be a challenge.

For this case study, busy hospital workers were required to detach themselves from their daily operational routine in order to get into the reflective mode of the interview and this proved difficult. During the semi-structured interviews, the participants required repeated re-direction to the topic by the interviewer (SSI 1 & SSI2). Qu [31] demonstrates how this can jostle with the interviewees thoughts:

"the flow of the interviewee’s story can be inadvertently disrupted by the interviewer, such as by redirecting the narrative or interrupting it… " p.248

In comparison, during the critical incident interviews, the interviewer guided participants towards one positive and one negative experience of a team intervention at a time. Rather than general impressions and trying to access their memory of the event, participants focussed on the specifics of that particular incident. In this way, use of CIT allowed the interviewee to focus their attention on telling their experiences of the positive or a negative incident, allowing detail to emerge organically whilst enabling the interviewer to probe for data that would support, refute or refine theories and/ or probe for new theories. This contributed to the quality, richness and usefulness of the data for refining the initial programme theory.

Little has been published on the efficiency of interviewing and the importance of this for healthcare workers in busy acute hospital contexts. Using the CIT, we found that as interviewers we could help participants access their memories of contextual conditions and outcomes in an efficient manner helping memories to re-surface by stimulating a “deep dive” on their thoughts as to how and why these things happened. Stimulating recall of specific incidents created an efficiency in the process. The average interview lasted forty-five minutes as opposed to over an hour for semi-structured interviews. In contrast to the semi-structured interviews, the majority of data retrieved had relevance for initial programme theory building, giving a concomitant efficiency when it came to the analysis phase for the researcher.

CIT literature [25] has demonstrated its evolving application “to focus more on thoughts, feelings, and why participants behaved as they did … in order to build on the practice of focusing on what a person did, why he/she did it, the outcome” (p490). In this study, we sought to understand the subjective reality of participants involved in team interventions in acute hospital contexts in order to gain insights into their motives, actions and intentions in a way that is meaningful. The critical incident interview format allowed interviewers to make connections between contexts (antecedents) and outcomes (consequences) of team interventions. Sometimes these emerged organically and on occasions through carefully embedded probes. Causal explanations of how and why team members choose as individuals and as a collective to behave the way they do under different contextual conditions could therefore be inferred. Realist researchers are often challenged trying to find mechanisms to connect contexts and outcomes. From the critical incident interviews, insights into generative causations were reasonably easy to develop. Participants on occasions unconsciously linked the contexts to outcomes and identified the mechanisms enacted themselves in these situations.

CIT therefore aligns well to Pawson’s theory driven Teacher-Learner style interviews [7] and complements Manzano’s [8] interview methods which are more directive and tailored towards theory evaluation or testing. The advantage of CIT has previously been cited [32] as “its capacity to explore differences or turning points; its utility as both a foundational/exploratory tool in the early stages of research and its role in building theories or models” (p480). CIT is particularly “suited to the exploration of dilemmas or looking at two sides of behaviour—good and bad; effective and ineffective; avoidable and unavoidable …” [11] p102. During critical incident interviews, interviewers extracted information from KIs without judgment. By inviting participants to focus on both negative and positive experiences of team interventions, CIT allowed a balanced approach to data collection. Researchers rarely publish detail of failed interventions and this may account for why barriers to team interventions had not previously been extrapolated from the literature. By drawing on negative experiences, a pattern of evidence emerged to support a theory relating to Inter-professional tensions and how and why they might impact on the success of team interventions. (CMOC 7).

Realist methods are underpinned by a critical realism philosophy. Manzano suggests that this ontological view will influence epistemology and choices in research design [8]. Although individuals’ reasoning cannot be seen, and has no external reality, these social phenomena have important practical application for team interventions. Understanding the reasons for certain phenomena in the form of an initial programme theory is a precursor to recommending a change in approach. Using CIT in this case study allowed identification of these causal explanations of regularities and irregularities in the social phenomena associated with team interventions. Whilst the interpretative approach of CIT is in keeping with the realist interpretative approach, the application of CIT in realist methods to date had not been explored.

Strengths

For staff in busy acute hospital contexts, variables pertaining to team interventions for example the composition of the team completing the intervention, team dynamics, team communication or organisational supports are rarely considered. This study demonstrates how use of CIT within realist methods allowed participants to consider these themes by channelling their recall towards specific incidents and the antecedents and consequences of those incidents, thus contributing to building an initial programme theory. The successful application and feasibility of the use of CIT for this purpose and in this setting is an important finding.

Manzano [8] encourages researchers to develop and share knowledge pertaining to the craft of interviewing. The results of this study demonstrate that CIT can effectively be applied within a realist methods framework. Authors have tried to demonstrate how they have respected the procedural integrity of CIT while embracing its inherent flexibility [18]. By sharing our experience of the use of CIT in developing an initial programme theory and developing a guide for the realist researcher, (Please refer to Appendix 2), we hope this provides some practical advice and guidance for researchers who may face similar challenges.

Limitations

In this case study, CIT was used to interview a purposively sampled group of hospital workers. The characteristics of this homogenous group from one hospital group in Ireland which had recently engaged in improvement work may have influenced their propensity to recall relevant material. Future research might therefore explore the application of CIT in realist methods in other healthcare contexts or with healthcare staff not involved in improvement work.

Conclusions

It was possible to adapt Flanagan’s five CIT procedures as a technique to help refine an initial programme theory. In particular, the CIT helped to stimulate clearer memories with regard to antecedents and consequences of specific team interventions and enabled data to be collected in a time efficient manner. CIT offered a balanced method of eliciting data from participants, allowing detail to emerge organically whilst still allowing flexibility for interviewers to probe for data relating to previous theories extrapolated from the literature. The CIT procedure should therefore be considered for use more widely within realist methodology and the authors have developed a practical guide to support other researchers in application of this approach.