Skip to main content
Download PDF

A scoping review of the problems and solutions associated with contamination in trials of complex interventions in mental health

BMC Medical Research Methodology volume 19, Article number: 4 (2019) Cite this article

Abstract

Background

In a randomised controlled trial, contamination is defined as the receipt of active intervention amongst participants in the control arm. This review assessed the processes leading to contamination, its typical quantity, methods used to mitigate it, and impact of use of cluster randomisation to prevent it on study findings in trials of complex interventions in mental health.

Methods

This is a scoping review of trial design approaches and methods of study conduct to address contamination. Studies included were randomised controlled trials of complex interventions in mental health that described the process leading to, amount of, or solution used to counter contamination. The Medline, Embase, and PsycInfo databases were searched for trials published between 2000 and 2015. Risk of bias was assessed using the Jadad score and domains recommended by Cochrane plus some relevant to cluster randomised trials.

Results

Two hundred and thirty-four articles were included in the review. The main processes that led to contamination were health professionals delivering both active and comparator treatments and communication among clinicians and participants from the different trial arms. Twenty-three trials (10%) measured binary treatment receipt in the control arm with median 13% of participants found to be contaminated (IQR 5–33%). The most common design approach for dealing with contamination was the use of cluster randomisation (n = 93). In addition, many researchers used simple trial conduct methods to minimise contamination due to suspected contamination processes, such as organising for each clinician to provide only one treatment and separating trial arms spatially or temporally. There was little evidence for a relationship between cluster randomisation to avoid contamination and size of treatment effect estimate.

Conclusion

There was some evidence of modest levels of treatment contamination with a large range, although a minority of studies reported the amount of contamination. A limitation was that many trials described the problem in little detail. Overall there is a need for greater measurement and reporting of treatment receipt in the control arm of trials. Researchers should be aware of trial conduct methods that can be used to minimise contamination without resorting to cluster randomisation.

Peer Review reports

Background

Treatment contamination is defined as the receipt of active intervention amongst participants in the control arm of a randomised controlled trial (RCT) [1]. It is thought to be particularly prevalent in RCTs of complex interventions in mental health. Psychological therapies are complex interventions that comprise several interacting constituent parts [2]. Such intervention components are often transportable and difficult to confine, meaning that their receipt by participants within the control arm is possible. The effect of contamination is to make the control arm more similar to the active intervention arm, i.e. to dilute the treatment contrast. This is a concern to researchers because the contrast between the randomised groups (intention-to-treat estimator) will be biased for the effect of treatment receipt (efficacy).

The processes leading to contamination in trials of complex interventions in mental health have never been reviewed comprehensively and the literature is unclear about their relative frequencies. This is necessary in order to plan what steps should be taken to address the problem. In mental health the typical quantity of contamination, some of the methods that researchers take to minimise or prevent it, and the extent to which it impacts on study findings are either little known or poorly formalised within the literature. Here we undertake a comprehensive scoping review that addresses these points. This type of review is defined as a map of literature within a research area that identifies key concepts; gaps in research; and types and sources of evidence, in order to inform practice, policymaking and research [3].

Previous reviews have assessed the extent of contamination in certain areas of medicine. For example, a literature review of 235 RCTs of guideline dissemination and implementation strategies for healthcare professionals identified eight trials that quantified contamination [1]. The review assessed the proportion of participants in the control arm who were considered to have received treatment and found a median of 24% of participants to be contaminated (range 0–65%). In oncology, a large breast cancer screening trial (n = 9780) found that 22% of those in control arm received a mammogram outside the trial compared to 5% of the intervention group doing likewise [4]. A review of cancer trials using Zelen’s design, where patients consent to their randomly allocated treatment before being asked for consent to participate in the study, found 11 trials that reported the number of patients who switched treatments [5, 6]. The average was 18% (range 10–36%). However, this figure did not represent solely contamination as many of the studies in the review described switches from active to comparator treatments or provided an overall summary of switches in either direction. The scale of the problem in mental health trials remains unclear.

In terms of steps that might be undertaken to minimise the occurrence of contamination, we distinguish between statistical design methods, trial conduct solutions, and analytical approaches. The main statistical design method is the use of cluster randomisation, which can prevent contamination provided that clusters are constructed at the level at which it takes place [7]. By ensuring that all participants within a cluster receive the same treatment, contamination of the control condition due to participants being affected by each other’s treatment receipt can be avoided. However, the cost of such a design is that correlation within clusters must be factored into a power calculation and will inflate the sample size requirement. Sample size is inflated by design factor, D.

$$ D:= 1+I\left(k-1\right) $$

where I is the intraclass correlation coefficient and k is the cluster size. In addition, cluster randomised controlled trials (cRCTs) often suffer from selection biases, mainly due to treatment being known before participant entry into trial (recruitment bias) and differential loss to follow-up between trial arms (attrition bias) [8]. The second type of strategy (trial conduct solutions) relates to methods that can be used in the running of the trial to reduce exposure of the control arm to active intervention. Education of clinicians and participants against contamination and provision of clear information about the purposes of the trial have been suggested [1]. However, it remains an open question as to what methods researchers use in practice. In terms of analytical methods that adjust for contamination, the use of modern causal estimation techniques has been advocated for this purpose [9,10,11], but it is not known how widely these have been used.

The review of educational interventions in RCTs also assessed whether there was any link between the prevention of contamination (e.g. through use of cluster randomisation) and an increase in the size of treatment effect estimates [1]. When assessing all studies in the set there was no evidence for such a link; however, a more homogeneous sample showed some evidence of a relationship. Other reviews have found similarly mixed results. For example, a review of 14 hip protector trials showed large positive effects in cluster RCTs and a mixture of positive and negative effects in RCTs with individual-level randomisation with suspected contamination [12]. On the other hand, a meta-analysis of 34 RCTs of enhanced care for depression found very similar treatment effect sizes when comparing cRCTs with individual-randomised RCTs [13]. One particular statistical design approach may provide extra information about the link between contamination and estimated treatment effect sizes. Specifically, in trials that use treatment allocation at more than one level, a comparison of treatment effect estimates between cluster- and participant-randomised sub-trials may provide some information as to the impact of contamination or the ability of cluster randomisation to prevent it.

The aims of this article were fourfold: to identify the processes that are considered to lead to contamination in trials of complex interventions in mental health, to quantify typical levels of contamination, to summarise what researchers do in order to prevent or mitigate it, and to compare treatment effect estimates within trials of complex interventions that used both cluster- and individual-level treatment allocation to quantify the contamination bias.

Methods

Type of review

We carried out a scoping review of trial design and conduct methods in RCTs of complex interventions in mental health. This type of review was chosen on the basis that our objectives were to summarise researchers’ perceptions of and solutions to a trial design problem where there is limited literature and potentially highly heterogeneous evidence. This was a methodology review and did not focus on a particular patient outcome, therefore was not eligible for registration with PROSPERO.

Eligibility criteria

All articles were screened using full texts and were assessed using five inclusion criteria. First, the text described a trial purporting to have used random allocation. Second, the intervention was complex, which in this review meant it comprised multiple components. It was not possible to assess whether these elements acted together to provide some added benefit (as per MRC guidance definition) so we used a general and therefore wide definition for this. Third, the publication gave some information about the process leading to, amount of, or solution used to counter treatment contamination. Fourth, the abstract and main body of the article were written in English. And finally, the trial was related to mental health, psychology, or psychiatry – this meant that a minimum of one of the target population, intervention, or primary outcome was directly related to one of these fields. Many trials in these fields test unblinded treatments where the suspicion is that they may be subject to contamination. The scoping review was limited to these areas of medicine for this reason and because of the apparent gap in the literature surrounding contamination in these fields.

Information sources

The search for contamination in RCTs of complex interventions in mental health was done using the Ovid platform and included the databases Medline, Embase, and PsycInfo. Articles that were published between January 2000 and April 2015 were searched. Results were restricted to those articles published after 2000 because this was the year when the first MRC framework paper on complex interventions was first published [14]. The publication of this framework marked the point at which the design and evaluation of complex interventions were formalised.

Search

Randomised controlled trials were searched for using the sensitivity-maximising 11-step process recommended by Cochrane [15]. The search terms “contamination” and “spillover” were included in the procedure. Synonymous terms for complex interventions that were used included all combinations of “multicomponent”, “psychosocial”, and “behavioural”, with “interventions”, “treatments”, and “training”. The search was restricted to articles that mentioned “mental health”, “psychology”, or “psychiatry”. All terms were searched for in the main body of the text. The full search procedure can be found in the supplementary materials (see file Additional file 1).

Study selection

Duplicates were removed from the set and the remaining articles were assessed for each of the exclusion criteria. Any potentially relevant article that was referred to by a paper in the results of the search and was not already in the set was followed up by a single author (NM). If the article was judged to have met the inclusion criteria it was included in the set and the full text was reviewed, also by a single author (NM). In order to assess the reliability of study selection, a second reviewer (RK) re-screened 70 articles (11%).

Data collection process

Any studies that were included in the review that featured sub-studies that used both cluster- and individual-level treatment allocation were reviewed as two separate sub-trials because of the different contamination processes and methods used to address these. Treatment effect sizes were extracted for trials that reported effects separately depending on the level of treatment allocation. Data from any such studies that did not report results at the different levels of treatment allocation were obtained from the authors in order to allow the comparison.

Data items

Abstracted data included an assessment of bias, summaries of trial design (e.g. study population, intervention, primary outcome, unit of treatment allocation), details about contamination (e.g. how it was thought to take place, its quantity, steps taken to avoid it), and records of trial summaries (e.g. extent of clustering, power, sample size, treatment effect). In order to assess the reliability of data abstraction, a second reviewer (RK) re-extracted data from 20 articles (8%) using the same procedure described above.

Risk of bias in individual studies

The review of trial bias included recording the “Jadad score” (a single item measure of methodological quality of RCTs [16]) and most of the domains of Cochrane’s classification scheme for bias [17]. In addition to these, some other domains that were pertinent to cluster randomised trials were used. These included whether randomisation occurred after participant consent was obtained, baseline measures were completed before randomisation, baseline outcome measurements were similar across trial arms, other clinical and demographic characteristics were similar across arms, and whether attrition was similar in the arms. These additional assessments of bias were based on outcomes used in a review of cRCTs [8].

Results

Reliability

At the screening stage agreement was 71%; all discrepancies were discussed and subsequently resolved. Agreement was 81% for all assessments of bias, and 82% for details of contamination processes.

Summary of trials

Two hundred and thirty-four studies were identified as meeting the eligibility criteria. This included seven trials that were referred to by an article in the main search and were found to meet the eligibility criteria. The results of the implementation strategy and numbers of exclusions are summarised in Fig. 1. Four hundred and fourteen articles were excluded. A list of these articles together with reasons for exclusions can be found in the supplementary materials (see Additional file 2).

Fig. 1
figure1

Flow diagram for searching for relevant articles (articles could be excluded for more than one reason)

Full size image

Details of the 234 trials in the review are given in Table 1. The table shows that the overwhelming majority of articles described the primary analysis of an RCT, were based in either North America or Western Europe, and were late phase (i.e. not pilot or feasibility trials). Most target populations were adult patients and the most commonly targeted conditions were depression, substance abuse, and psychosis. The two most common interventions were cognitive behavioural therapy and care management; there were many small categories. The sample size of participant randomised trials ranged from 16 to 14,910; that of cluster randomised trials ranged from 13 to 6076. A full list of references can be found in the supplementary materials (see Additional file 3).

Table 1 Summary of characteristics of articles
Full size table

Summary of assessment of bias

Summaries of assessments of bias using the Jadad scale, items recommended by Cochrane, and items aimed at identifying possible bias in cluster randomised trials are reported in Table 2. The table demonstrates the potential for bias split by the level of treatment allocation. In general, it shows that the greatest potential for bias arose due to lack of knowledge of allocation, incomplete outcome data, differences in attrition between trial arms, and randomisation occurring before consent was obtained and before baseline measures were completed. More evidence for bias was found in cluster randomised trials when assessing whether randomisation took place after consent and after baseline measures were completed, whether outcome assessment was blind, and whether baseline demographic characteristics were similar between trial arms. Assessments of bias for each trial together with the support for judgement associated with each assessment can be found in the supplementary materials (see Additional file 3).

Table 2 Summary of assessment of bias
Full size table

Processes driving contamination

There were perceived to be five main processes that led to contamination. The first two processes, staff delivering the active intervention in the control arm (n = 82, 35%) and communication between trial arms (n = 79, 34%), were the most common. Staff delivering the active intervention in the control arm happened either due to a given clinician delivering both the active and control treatments (n = 76, e.g. [18]) or due to control participants being exposed to the intervention as a consequence of clinicians, who were not directly involved in providing the treatment, treating participants in both arms and thereby potentially learning about the active intervention and passing this on to participants in the control arm (n = 6, e.g. [19]). The other main contamination process was communication between individuals in different trial arms. This could be either at the level of the clinician (n = 20, e.g. [20]), participant (n = 57, e.g. [21]), or both (n = 2). Communication between providers of interventions was often a worry in environments in which the people giving the treatment worked closely together, for example GP surgeries, hospital units, and schools. Communication between participants was thought to be most likely in environments in which participants came into close contact. Examples of this included interaction between participants who were family members, patients in a waiting room, school children, employees working on the same worksite, and university students. Particular healthcare settings that were thought to be highly likely to foster communication were antenatal clinics/childbirth classes, specialist clinics (e.g. substance misuse, dialysis), and wards for those admitted to hospital.

There were perceived to be three other, more minor processes that drove contamination. First, participants switching clinicians (n = 4, 2%, e.g. [22]), where control participants were treated by multiple clinicians of whom one was trained in the active intervention. Second, participants seeking treatment outside the trial (n = 6, 3%, e.g. [23]). And finally, background noise, where the treatment already existed to some extent within the healthcare system (n = 5, 2%, e.g. [24]). Fifty-nine articles did not provide information about the contamination process.

Quantity of contamination

Twenty-seven studies (12%) attempted to quantify contamination. Twenty-three trials measured individual-level contamination on a binary scale and summaries of these quantities are given in Table 3. The median level of contamination was 13% (IQR 5–33%).

Table 3 Quantifying treatment contamination where treatment receipt was defined as binary
Full size table

Four trials measured contamination using a continuous scale; three were trials of cognitive behavioural therapy and one of cognitive analytic therapy. One created a treatment fidelity scale and asked participants in each trial arm (behavioural weight control instructions, cognitive behavioural therapy, standard counselling) about their knowledge of all three treatments at the beginning and end of treatment [25]. The sub-scales showed high knowledge of behavioural weight control in the group allocated to receive behavioural weight control instructions (mean change of 1.1 compared to 0.5 and 0.5 in cognitive behavioural therapy and standard arms), high knowledge of cognitive behavioural therapy in those allocated to receive this (mean change of 1.6 compared to 0.0 and 0.8 in behavioural weight control and standard groups), and high knowledge of standard intervention in the control group (mean change of 0.5 compared to 0.1 and 0.1 in behavioural weight control and cognitive behavioural therapy arms). This seemed to indicate receipt of treatment in the control arm. Three RCTs showed negligible evidence of treatment contamination. Of these RCTs, one used a cognitive behavioural therapy adherence scale (adapted CTACS) to record adherence and contamination in the active intervention and control arms [26]. The CTACS means were 98.0 and 98.8 in the cognitive behavioural therapy and education intervention (control) arms respectively, indicating that contamination did not occur. Another trial found that the family-focused cognitive behavioural therapy (FCBT; active intervention) group scored higher than the traditional child-focused cognitive behavioural therapy (control) group on two scales, Family Focus (mean = 4.90 and 1.55) and Parenting Style Focus (mean = 4.75 and 1.00) [27]. This suggested that only the FCBT group incorporated family and parenting interventions and therefore that there was little evidence of contamination. The fourth trial used a scale to measure the fidelity of the control intervention, which was good clinical care [28]. This scale included a sub-scale for cognitive analytic therapy and the mean for this was very low: 0.52 (SD 0.11). This represented negligible contamination.

Solutions used to counter contamination

Methods that were used to counter contamination are summarised in three categories: statistical design, trial conduct, and analysis methods. Statistical design includes the use of cluster randomisation, where clusters are chosen based on groups of participants who are thought potentially to become contaminated by direct or indirect links (e.g. via a shared therapist). One trial inflated the sample size in order to account for reduced statistical power caused in part by contamination bias [29]. The great majority of other methods for preventing contamination were aspects of trial conduct, such as recruitment of more clinicians to ensure that each clinician only delivered one of the interventions. In terms of analysis methods, one trial used per protocol analysis, meaning that participants whose treatment was contaminated were dropped from the analysis [30]. This review found no trials that addressed the problem of contamination by using methods from the causal inference field.

Categorisations of trial conduct solutions that were used to avoid treatment contamination can be found in Table 4. The sections of the table match the processes of contamination described in the earlier section on this. The majority of solutions used to prevent contamination related either to preventing staff delivering the active intervention in the control arm or preventing communication between clinicians or participants.

Table 4 Trial conduct solutions to treatment contamination
Full size table

Four trials were concerned about contamination during data collection and aimed to prevent this by minimising interaction between researchers and participants [31,32,33,34]. Another temporally separated the control and active treatments with data collection following each. This meant that treatment could only influence data from active intervention participants [35].

Trials using both cluster- and participant-level treatment allocation

The results of the review included four trials that used both participant- and cluster-level treatment allocation [36,37,38,39]. Treatment effect estimates and confidence intervals for these trials are shown in Fig. 2. The figure shows treatment effects arranged such that greater benefit (or less harm) of treatment is represented by a greater number on the horizontal axis. The figure enables the comparison of the absolute size of treatment effect between participant- and cluster-level allocation to assess the impact of contamination on effect size estimation. Of the 21 outcomes investigated, just under half of outcomes showed a difference in the anticipated direction, i.e. smaller estimated absolute effect sizes under participant-level random allocation. In particular, an attenuated treatment effect (lesser distance from the null line in Fig. 2) was found under participant-level allocation in eight out of 21 outcomes with a tie in one outcome.

Fig. 2
figure2

Forest plots for four trials that used both individual- and cluster-level randomisation; (P) = primary outcome. a) Clarkson et al. (2009) [36]. Choice of primary outcomes is based on sample size calculation; estimates are adjusted for baseline measures. Larger (more positive) treatment effects indicate benefit. b) Lee & Gay (2011) [37]. Estimates were standardised and calculated from summaries of means and SDs (mothers’ scores only). Larger (more positive) treatment effects indicate benefit. c) Marshall et al. (2004) [38]. Estimates used same adjustments as in the trial publication. Larger (more positive) treatment effects indicate benefit. d) Richards et al. (2008) [39]. Estimates were standardised and calculated from summaries of means and SDs. Larger (more positive) treatment effects indicate benefit

Full size image

Discussion

The review identified 234 articles that described either the processes driving treatment contamination, its quantity, or steps that researchers took to prevent or alleviate the problem in trials of complex interventions in mental health. The principal processes leading to contamination were found to be clinicians being required to treat participants in both treatment and control conditions and communication among clinicians or participants in different trial arms. Typically, around one in eight participants in the control arm of a trial were assessed as having received the active intervention. The most common steps that researchers took to prevent or mitigate contamination were the use of cluster randomisation, organising for each clinician to provide only one type of treatment, monitoring treatment receipt, spatially or temporally separating trial arms, and informing participants about only the treatment that they were allocated to receive. There was little evidence of a difference in the magnitude of treatment effects within trials that used both cluster- and participant-level treatment allocation.

The classification of two main processes and three more minor types of contamination was based on the processes that researchers and clinicians described in such trials. The main trial conduct steps that researchers took to minimise contamination were in line with the processes that were found to be driving it. Many researchers attempted to design against contamination by carefully controlling the treatment’s delivery. These processes were often anticipated and then prevented or attenuated by the designers of the trials. There were no examples of researchers first having evaluated in detail treatment receipt within the control arm. The processes described here therefore partly represent researchers’ expectations and not entirely clinician or participant behaviour.

The small number of trials that measured and reported treatment receipt in the control arm found it to be affecting a minority of the control participants. The distribution of this was similar to the quantity found previously in other areas of medicine such as educational interventions [1], breast cancer screening [4], and cancer trials using Zelen’s design [5]. Thus while there is a lot of concern about contamination it is not clear that this problem is indeed widespread. The extent of the problem may be related to the complexity, intensity, and nature of the intervention.

Researchers often used cluster randomisation to prevent treatment contamination, amongst other reasons. While cRCTs can avoid contamination bias they are at risk of other biases. Our set of articles included 93 cluster randomised trials. Assessments of bias suggested that cRCTs were more likely to be affected by bias when considering whether randomisation took place after consent was obtained and after baseline measures were completed, whether outcome assessment was blind, and whether demographic characteristics were similar between trial arms. This was consistent with an earlier review of cRCTs that were published in three prominent medical journals which found evidence of recruitment bias [8].

The small number of trials that allocated treatment at both cluster and participant levels did not find any evidence for differences in effect size estimates. The lack of evidence for a link between the level of randomisation and treatment effect size suggested that either the employment of cluster randomisation did not prevent contamination, the anticipated contamination was overstated, or that the use of cluster randomisation led to a similar degree of bias as that caused by contamination in the participant-randomised trials. Overall, the finding was consistent with those of a review of trials of enhanced care in depression [12], and of educational interventions [1]. Similarly to previous reviews, there was considerable heterogeneity between trials identified in this study that used both cluster- and participant-level randomisation. However, the variability here is between trials and not within them because randomisation implies that the sub-trials were balanced for every variable except the level at which treatment allocation took place. It is possible that the impacts of contamination and cluster randomisation on bias are dependent on the disease or type of intervention. In order to draw substantive conclusions about the effect of treatment allocation level on contamination, a systematic review of this particular trial design is needed.

Conclusions

This is the most comprehensive review of contamination in mental health trials to date. It is the first to identify the processes leading to contamination and the measures that researchers take in order to minimise the problem. The main limitation is that the trials were heterogeneous in that they represented a large range of illnesses and interventions. With regard to the causes of contamination, it is an assumption that the processes described by authors were the drivers of contamination. Another limitation is the time interval between the literature search and publication of this review. It is possible that the problems, solutions and reporting of contamination have evolved since the search took place.

The results of this review suggest that treatment contamination is perceived to be a significant problem in trials of complex interventions in mental health. However, the trials that measured and reported it suggest that the phenomenon is often modest (with a large range). The reporting of it is infrequent and almost certainly not as commonplace as that of treatment non-compliance. This implies a need for greater measurement and reporting of treatment receipt in the control arm of trials in this field. The findings also show that there are many steps that researchers can take to minimise contamination without resorting to the use of cluster randomisation. In addition, we found that modern causal analysis methods, including the techniques developed particularly for contamination adjustment [9, 10], are yet to be utilised to deal with contamination bias at the analysis stage. This is likely to be in part a reflection of the infrequency of measurement of treatment receipt for all participants in the control trial arm.

Abbreviations

CBT:

cognitive behavioural therapy

cRCT:

cluster randomised controlled trial

CTACS:

cognitive behavioural therapy adherence scale

FCBT:

family-focused cognitive behavioural therapy

IQR:

interquartile range

MRC:

Medical Research Council

RCT:

randomised controlled trial

References

  1. 1.

    Keogh-Brown M, Bachmann M, Shepstone L, Hewitt C, Howe A, Ramsay C, Song F, Miles J, Torgerson D, Miles S, et al. Contamination in trials of educational interventions. Health Technol Asses. 2007;11(43):1–130.

    Article  Google Scholar 

  2. 2.

    Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.

    Article  Google Scholar 

  3. 3.

    Daudt HM, van Mossel C, Scott SJ. Enhancing the scoping study methodology: a large, inter-professional team's experience with Arksey and O'Malley's framework. BMC Med Res Methodol. 2013;13:48.

    Article  Google Scholar 

  4. 4.

    Goel V, Cohen MM, Kaufert P, MacWilliam L. Assessing the extent of contamination in the Canadian National Breast Screening Study. Am J Prev Med. 1998;15(3):206–11.

    CAS  Article  Google Scholar 

  5. 5.

    Torgerson DJ. Contamination in trials: is cluster randomisation the answer? BMJ. 2001;322(7282):355–7.

    CAS  Article  Google Scholar 

  6. 6.

    Altman DG, Whitehead J, Parmar MK, Stenning SP, Fayers PM, Machin D. Randomised consent designs in cancer clinical trials. Eur J Cancer. 1995;31A(12):1934–44.

    CAS  Article  Google Scholar 

  7. 7.

    Hayes R, Moulton L. Cluster randomised trials: chapman and hall/CRC biostatistics series; 2009.

    Google Scholar 

  8. 8.

    Puffer S, Torgerson D, Watson J. Evidence for risk of bias in cluster randomised trials: review of recent trials published in three general medical journals. BMJ. 2003;327(7418):785–9.

    Article  Google Scholar 

  9. 9.

    Cuzick J, Edwards R, Segnan N. Adjusting for non-compliance and contamination in randomized clinical trials. Stat Med. 1997;16(9):1017–29.

    CAS  Article  Google Scholar 

  10. 10.

    Dunn G, Maracy M, Tomenson B. Estimating treatment effects from randomized clinical trials with noncompliance and loss to follow-up: the role of instrumental variable methods. Stat Methods Med Res. 2005;14(4):369–95.

    Article  Google Scholar 

  11. 11.

    Dunn G, Emsley R, Liu HH, Landau S, Green J, White I, Pickles A. Evaluation and validation of social and psychological markers in randomised trials of complex interventions in mental health: a methodological research programme. Health Technol Asses. 2015;19(93).

  12. 12.

    Hahn S, Puffer S, Torgerson DJ, Watson J. Methodological bias in cluster randomised trials. BMC Med Res Methodol. 2005;5:10.

    Article  Google Scholar 

  13. 13.

    Gilbody S, Bower P, Torgerson D, Richards D. Cluster randomized trials produced similar results to individually randomized trials in a meta-analysis of enhanced care for depression. J Clin Epidemiol. 2008;61(2):160–8.

    Article  Google Scholar 

  14. 14.

    Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321(7262):694–6.

    CAS  Article  Google Scholar 

  15. 15.

    Lefebvre C, Manheimer, E, Glanville, J: Chapter 6: Searching for studies. In: Cochrane Handbook for Systematic Reviews of Interventions Version 5 1 0. edn. Edited by Higgins J, Green S: The Cochrane Collaboration; 2011.

  16. 16.

    Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17(1):1–12.

    CAS  Article  Google Scholar 

  17. 17.

    Higgins J, Green S (eds.): Cochrane Handbook for Systematic Reviews of Interventions Version 5 1 0 [updated March 2011]: Cochrane Collaboration; 2011.

  18. 18.

    Barkhof E, Meijer CJ, de Sonneville LMJ, Linszen DH, de Haan L. The effect of motivational interviewing on medication adherence and hospitalization rates in nonadherent patients with multi-episode schizophrenia. Schizophr Bull. 2013;39(6):1242–51.

    Article  Google Scholar 

  19. 19.

    Beck CK, Vogelpohl TS, Rasin JH, Uriri JT, O'Sullivan P, Walls R, Phillips R, Baldwin B. Effects of behavioral interventions on disruptive behavior and affect in demented nursing home residents. Nursing Research July/August. 2002;51(4):219–28.

    Article  Google Scholar 

  20. 20.

    Johnson S, Thornicroft G, Afuwape S, Lesse M, Hughes E, Waingarante S, Miles H, Craig T. Effects of training community staff in interventions for substance misue in dual diagnosis patients with psychosis (COMO study): cluster randomised trial. Br J Psychiatry. 2007;191:451–2.

    CAS  Article  Google Scholar 

  21. 21.

    Ersek M, Turner J, Cain K, Kemp C. Results of a randomized controlled trial to examine the efficacy of a chronic pain self-management group for older adults [ISRCTN11899548]. Pain. 2008;138(1):29–40.

    Article  Google Scholar 

  22. 22.

    Cooper LA, Ghods Dinoso BK, Ford DE, Roter DL, Primm AB, Larson SM, Gill JM, Noronha GJ, Shaya EK, Wang N-Y. Comparative effectiveness of standard versus patient-centered collaborative care interventions for depression among African Americans in primary care settings: the BRIDGE study. Health Serv Res. 2013;48(1):150–74.

    Article  Google Scholar 

  23. 23.

    Stuifbergen AK, Blozis SA, Becker H, Phillips L, Timmerman G, Kullberg V, Taxis C, Morrison J. A randomized controlled trial of a wellness intervention for women with fibromyalgia syndrome. Clin Rehabil. 2010;24(4):305–18.

    Article  Google Scholar 

  24. 24.

    Becona E, Vazquez FL. Effectiveness of personalized written feedback through a mail intervention for smoking cessation: a randomized-controlled trial in Spanish smokers. J Consult Clin Psychol. 2001;69(1):33–40.

    CAS  Article  Google Scholar 

  25. 25.

    Perkins KA, Marcus MD, Levine MD, D'Amico D, Miller A, Broge M, Ashcom J, Shiffman S. Cognitive-behavioral therapy to reduce weight concerns improves smoking cessation outcome in weight-concerned women. Consult Clin Psychol. 2001;69(4):604–13.

    CAS  Article  Google Scholar 

  26. 26.

    Thorn BE, Day MA, Burns J, Kuhajda MC, Gaskins SW, Sweeney K, McConley R, Ward CL, Cabbil C. Randomized trial of group cognitive behavioral therapy compared with a pain education control for low-literacy rural people with chronic pain. Pain. 2011;152(12):2710–20.

    Article  Google Scholar 

  27. 27.

    Wood JJ, Piacentini JC, Southam-Gerow M, Chu BC, Sigman M. Family cognitive behavioral therapy for child anxiety disorders. J Am Acad Child Adolesc Psychiatry. 2006;45(3):314–21.

    Article  Google Scholar 

  28. 28.

    Chanen AM, Jackson HJ, McCutcheon LK, Jovev M, Dudgeon P, Yuen HP, Germano D, Nistico H, McDougall E, Weinstein C, et al. Early intervention for adolescents with borderline personality disorder using cognitive analytic therapy: randomised controlled trial. Br J Psychiatry. 2008;193(6):477–84.

    Article  Google Scholar 

  29. 29.

    Dobscha SK, Corson K, Perrin NA, Hanson GC, Leibowitz RQ, Doak MN, Dickinson KC, Sullivan MD, Gerrity MS. Collaborative care for chronic pain in primary care: a cluster randomized trial. JAMA. 2009;301(12):1242–52.

    CAS  Article  Google Scholar 

  30. 30.

    Pfiffner LJ, Yee Mikami A, Huang-Pollock C, Easterlin B, Zalecki C, McBurnett K. A randomized, controlled trial of integrated home-school behavioral treatment for ADHD, predominantly inattentive type. J Am Acad Child Adolesc Psychiatry. 2007;46(8):1041–50.

    Article  Google Scholar 

  31. 31.

    Chan MF, Ng SE, Tien A, Man Ho RC, Thayala J. A randomised controlled study to explore the effect of life story review on depression in older Chinese in Singapore. Health Soc Care Community. 2013;21(5):545–53.

    Article  Google Scholar 

  32. 32.

    Chochinov HM, Kristjanson LJ, Breitbart W, McClement S, Hack TF, Hassard T, Harlos M. Effect of dignity therapy on distress and end-of-life experience in terminally ill patients: a randomised controlled trial. Lancet Oncol. 2011;12(8):753–62.

    Article  Google Scholar 

  33. 33.

    McLaughlin TJ, Aupont O, Bambauer KZ, Stone P, Mullan MG, Colagiovanni J, Polishuk E, Johnstone M, Locke SE. Improving psychologic adjustment to chronic illness in cardiac patients: the role of depression and anxiety. J Gen Intern Med. 2005;20(12):1084–90.

    Article  Google Scholar 

  34. 34.

    Tiwari A, Leung WC, Leung TW, Humphreys J, Parker B, Ho PC. A randomised controlled trial of empowerment training for Chinese abused pregnant women in Hong Kong. BJOG Int J Obstet Gynaecol. 2005;112(9):1249–56.

    CAS  Article  Google Scholar 

  35. 35.

    Alessi CA, Martin JL, Webber AP, Kim CE, Harker JO, Josephson KR. Randomized, controlled trial of a nonpharmacological intervention to improve abnormal sleep/wake patterns in nursing home residents. J Am Geriatr Soc. 2005;53(5):803–10.

    Article  Google Scholar 

  36. 36.

    Clarkson JE, Young L, Ramsay CR, Bonner BC, Bonetti D. How to influence patient oral hygiene behavior effectively. J Dent Res. 2009;88(10):933–7.

    CAS  Article  Google Scholar 

  37. 37.

    Lee KA, Gay CL. Can modifications to the bedroom environment improve the sleep of new parents? Two randomized controlled trials. Res Nurs Health. 2011;34(1):7–19.

    Article  Google Scholar 

  38. 38.

    Marshall M, Lockwood A, Green G, Zajac-Roles G, Roberts C, Harrison G. Systematic assessments of need and care planning in severe mental illness: cluster randomised controlled trial. Br J Psychiatry. 2004;185:163–8.

    CAS  Article  Google Scholar 

  39. 39.

    Richards D, Lovell K, Gilbody S, Gask L, Torgerson D, Barkham M, Bland M, Bower P, Lankshear A, Simpson A, et al. Collaborative care for depression in UK primary care: a randomized controlled trial. Psychol Med. 2008;38(2):279–87.

    CAS  Article  Google Scholar 

  40. 40.

    Aveyard P, Brown K, Saunders C, Alexander A, Johnstone E, Munafo MR, Murphy M. Weekly versus basic smoking cessation support in primary care: a randomised controlled trial. Thorax. 2007;62(10):898–903.

    Article  Google Scholar 

  41. 41.

    Barton MB, Morley DS, Moore S, Allen JD, Kleinman KP, Emmons KM, Fletcher SW. Decreasing women's anxieties after abnormal mammograms: a controlled trial. J Natl Cancer Inst. 2004;96(7):529–38.

    Article  Google Scholar 

  42. 42.

    Bernstein GA, Layne AE, Egan EA, Tennison DM. School-based interventions for anxious children. J Am Acad Child Adolesc Psychiatry. 2005;44(11):1118–27.

    Article  Google Scholar 

  43. 43.

    Borland R, Balmford J, Benda P. Population-level effects of automated smoking cessation help programs: a randomized controlled trial. Addiction. 2013;108(3):618–28.

    Article  Google Scholar 

  44. 44.

    Courneya KS, Friedenreich CM, Sela RA, Quinney H, Rhodes RE, Handman M. The group psychotherapy and home-based physical exercise (group-hope) trial in cancer survivors: physical fitness and quality of life outcomes. Psycho-Oncology. 2003;12(4):357–74.

    Article  Google Scholar 

  45. 45.

    Dilley JW, Woods WJ, Loeb L, Nelson K, Sheon N, Mullan J, Adler BL, Chen S, McFarland W. Brief cognitive counseling with HIV testing to reduce sexual risk among men who have sex with men: results from a randomized controlled trial using paraprofessional counselors. JAIDS. 2007;44(5):569–77.

    PubMed  Google Scholar 

  46. 46.

    Forchuk C, Martin ML, Chan YL, Jensen E. Therapeutic relationships: from psychiatric hospital to community. J Psychiatr Ment Health Nurs. 2005;12(5):556–64.

    CAS  Article  Google Scholar 

  47. 47.

    Heirich M, Sieck CJ. Worksite cardiovascular wellness programs as a route to substance abuse prevention. J Occup Environ Med. 2000;42(1):47–56.

    CAS  Article  Google Scholar 

  48. 48.

    Khumalo-Sakutukwa G, Morin S, Fritz K, Charlebois E, van Rooyen H, Chingono A, Modiba P, Mrumbi K, Visrutaratna S, Singh B, et al. Project accept (HPTN 043): a community-based intervention to reduce HIV incidence in populations at risk for HIV in sub-Saharan Africa and Thailand. JAIDS. 2008;49(4):422–31.

    PubMed  Google Scholar 

  49. 49.

    Lamers F, Jonkers C, Bosma H, Kempen G, Meijer J, Penninx B, Knottnerus A, van Eijk J. A minimal psychological intervention in chronically ill elderly patients with depression: a randomized trial. Psychother Psychosom. 2010;79(4):217–26.

    Article  Google Scholar 

  50. 50.

    Merritt RK, Price JR, Mollison J, Geddes JR. A cluster randomized controlled trial to assess the effectiveness of an intervention to educate students about depression. Psychol Med. 2007;37(3):363–72.

    Article  Google Scholar 

  51. 51.

    Moadel AB, Bernstein SL, Mermelstein RJ, Arnsten JH, Dolce EH, Shuter J. A randomized controlled trial of a tailored group smoking cessation intervention for HIV-infected smokers. JAIDS. 2012;61(2):208–15.

    PubMed  Google Scholar 

  52. 52.

    Mohr DC, Carmody T, Erickson L, Jin L, Leader J. Telephone-administered cognitive behavioral therapy for veterans served by community-based outpatient clinics. J Consult Clin Psychol. 2011;79(2):261–5.

    Article  Google Scholar 

  53. 53.

    Phillips G, Bottomley C, Schmidt E, Tobi P, Lais S, Yu G, Lynch R, Lock K, Draper A, Moore D, et al. Well London Phase-1: results among adults of a cluster-randomised trial of a community engagement approach to improving health behaviours and mental well-being in deprived inner-city neighbourhoods. J Epidemiol Community Health. 2014;68(7):606–14.

    Article  Google Scholar 

  54. 54.

    Saitz R, Cheng D, Winter M, Kim TW, Meli SM, Allensworth-Davies D, Lloyd-Travaglini CA, Samet JH. Chronic care management for dependence on alcohol and other drugs: the AHEAD randomized trial. JAMA. 2013;310(11):1156–67.

    CAS  Article  Google Scholar 

  55. 55.

    Shemilt I, Harvey I, Shepstone L, Swift L, Reading R, Mugford M, Belderson P, Norris N, Thoburn J, Robinson J. A national evaluation of school breakfast clubs: evidence from a cluster randomized controlled trial and an observational analysis. Child Care, Health Dev. 2004;30(5):413–27.

    CAS  Article  Google Scholar 

  56. 56.

    Stewart-Brown S, Patterson J, Mockford C, Barlow J, Klimes I, Pyper C. Impact of a general practice based group parenting programme: quantitative and qualitative results from a controlled trial at 12 months. Arch Dis Child. 2004;89(6):519–25.

    CAS  Article  Google Scholar 

  57. 57.

    Waghorn G, Dias S, Gladman BM, Harris M, Saha S. A multi-site randomised controlled trial of evidence-based supported employment for adults with severe and persistent mental illness. Aust Occup Ther J. 2014;61(6):424–36.

    Article  Google Scholar 

  58. 58.

    Walpole B, Dettmer E, Morrongiello BA, McCrindle BW, Hamilton J. Motivational interviewing to enhance self-efficacy and promote weight loss in overweight and obese adolescents: a randomized controlled trial. J Pediatr Psychol. 2013;38(9):944–53.

    Article  Google Scholar 

  59. 59.

    Wells DL, Dawson P, Sidani S, Craig D, Pringle D. Effects of an abilities-focused program of morning care on residents who have dementia and on caregivers. J Am Geriatr Soc. 2000;48(4):442–9.

    CAS  Article  Google Scholar 

Download references

Acknowledgments

Not applicable.

Funding

This report is independent research arising from a Doctoral Research Fellowship (DRF-2014-07-002) supported by the National Institute for Health Research (NIHR). It is also part funded by the NIHR Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health, UK.

Availability of data and materials

The datasets used and analysed during the current study are available from the corresponding author on reasonable request.

Author information

Affiliations

  1. Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, Kings College London, 16 De Crespigny Park, London, SE5 8AF, United Kingdom

    Nicholas Magill, Ruth Knight & Sabine Landau

  2. Department of Medical Statistics, London School of Hygiene and Tropical Medicine, London, UK

    Nicholas Magill

  3. Department of Health Service and Population Research, Institute of Psychiatry, Psychology, and Neuroscience, King’s College London, London, UK

    Paul McCrone

  4. Institute of Diabetes, Endocrinology and Obesity, King’s Health Partners, London, UK

    Khalida Ismail

Authors
  1. Nicholas Magill
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruth Knight
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul McCrone
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Khalida Ismail
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sabine Landau
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

NM performed the literature search and led the preparation of the final manuscript. SL, KI, and PM supervised the design of the research. RK repeated some of the screening for eligibility and data abstraction to allow an assessment of reliability. All authors provided input into the interpretation of the results and drafting of the article, and approved the final manuscript.

Corresponding author

Correspondence to Nicholas Magill.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Search procedure for Ovid platform. The file includes the steps used in the Ovid search procedure for trials of complex interventions in mental health where contamination was a problem are listed below. (DOCX 15 kb)

Additional file 2:

Studies excluded at full text screening. The file contains a list of references for all studies that were excluded at the screening stage together with reasons for exclusion. (DOCX 87 kb)

Additional file 3:

List of references with assessment of bias and support for judgement. The file contains a list of references for all studies that were included in the review together with the assessment of risk of bias and the support for judgement. (DOCX 323 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Magill, N., Knight, R., McCrone, P. et al. A scoping review of the problems and solutions associated with contamination in trials of complex interventions in mental health. BMC Med Res Methodol 19, 4 (2019). https://doi.org/10.1186/s12874-018-0646-z

Download citation

Keywords

  • Treatment contamination
  • Randomised controlled trials
  • Mental health
  • Complex interventions

BMC Medical Research Methodology

ISSN: 1471-2288

Contact us