Cell phone-supported cognitive behavioural therapy for anxiety disorders: a protocol for effectiveness studies in frontline settings
© Ekberg et al; licensee BioMed Central Ltd. 2011
Received: 13 August 2010
Accepted: 10 January 2011
Published: 10 January 2011
Reviews of randomized controlled trials (RCTs) of cognitive behavioural therapy (CBT) for anxiety disorders have reported large pre- to post-treatment within-group effect sizes on measures of anxiety when supplied in therapist consultations and in technology-supported settings. However, the stringent experimental control of RCTs results in a lack of external validity, which limits the generalizability of findings to real-world frontline clinical practice. We set out to examine the specification of a protocol for study of the effectiveness of cell phone-supported CBT for in situ management of anxiety disorders.
Methods and design
Nominal group methods were used for requirements analysis and protocol design. Making a distinction between different forms of technology-supported therapy, examination of therapists' role, and implementing trials in existing organizational and community contexts were found to be the central requirements in the protocol.
The resulting protocol (NCT01205191 at clinicaltrials.gov) for use in frontline clinical practice in which effectiveness, adherence, and the role of the therapists are analyzed, provides evidence for what are truly valuable cell phone-supported CBT treatments and guidance for the broader introduction of CBT in health services.
Ample evidence indicates that anxiety is under treated in western societies, and that large numbers of the population suffer needlessly. The World Health Organization (WHO) has estimated that 40% of disability attributed to anxiety reflects the fact that many potential anxiety patients never reach health care . A major factor contributing to this shortcoming is that evidence-based psychotherapies are not practiced widely in community settings [2, 3]. For instance, less than a quarter of those with anxiety disorders in the United Kingdom receive treatment of any sort . In the United States, only about 40% of patients with mood or anxiety disorders receive any treatment . It is in this context that expectations of cognitive behavioural therapy (CBT) are high. CBT methodology has the advantage of using well-defined treatment practices that can be easily taught to a variety of therapists and whose implementation can be monitored. Reviews of randomized controlled trials (RCTs) of CBT for anxiety disorders have reported large pre- to post-treatment within-group effect sizes on measures of anxiety when supplied in therapist consultations [6, 7], in computer-supported sessions at practices [8, 9], and over the Internet [10, 11]. In Sweden, CBT is the current treatment of choice for mild to moderate anxiety disorder .
Despite its proven efficacy, CBT still seems to be neglected in practice settings. In this study, we address 2 possible reasons for this situation. The first is that most of the research has been conducted using samples with isolated (rather than comorbid) disorders, but most therapists help individuals with multiple comorbid problems. It is not known if CBT techniques adapted to a particular client's needs by a skilled therapist in a community setting achieve a better result than a therapist following a structured routine. It does seem that experienced therapists prefer to select from a variety of techniques rather than to follow a regimented program. Moreover, patient non-adherence is a persistent and complex phenomenon in frontline CBT practice . The refractory nature of this problem is reflected by the fact that a large proportion of non-adherence has come to be routinely accepted when planning interventions. Critics argue that RCTs' stringent experimental controls (e.g. patients with homogenous diagnoses and highly trained and supervised therapists) result in a lack of external validity, which limits the generalizability of findings to real-world or frontline clinical practice [14, 15]. In addition, studies of the effectiveness of CBT for anxiety disorders have consistently reported lower patient adherence than RCTs [13, 16].
We conclude that there is a need to examine carefully the external validity and effectiveness of CBT treatments for anxiety that have already been shown to be efficacious, and to identify factors that reduce non-adherence. In this study, we used these presumptions to plan for the examination of a new generation of CBT programs. With this understanding, we set out to define a protocol for examination of the effectiveness of cell phone-supported CBT for anxiety disorders in frontline settings that also allows to evaluate adherence to therapy. Specifically, the aims of this study were to define a protocol for assessment of
the superiority of cell phone-supported CBT (CBT-cell phone) treatment against anxiety disorders compared with CBT treatment as usual (CBT-TAU), and
adherence with CBT-cell phone compared with CBT-TAU and CBT provided with access to a placebo technical device (CBT-placebo).
Methods and design
Requirements data collection
The task communicated to the therapy panel was to make sure the protocol conformed to organizational and therapeutic routines and content. The task communicated to the technical design panel was to make sure that the protocol conformed to present cell phone technology and limitations. In addition, the technical design panel, in cooperation with the therapy panel, developed a specification of the technical systems to ensure that, in accordance with the protocol, the content of the technical solution was compatible with the paper-based homework used in present CBT for anxiety syndrome.
Requirements data analysis
The data from the study requirement analysis were transferred to a protocol specification by the study protocol panel. The specification was developed during a series of meetings lasting for 2 hours each (n = 8) at which the study protocol, technical design, and therapy methods in use were systematically developed and discussed. The specification was finally refined at separate meetings with the therapy and technical design panels.
Study protocol requirements
The results of the requirement analysis yielded 3 requirement areas that were found to be important when studying the effectiveness and adherence to cell phone-supported CBT in a community context.
Even if different forms of CBT have been shown to be efficacious in RCTs, therapy provided in a community setting entails a wider variety of patient problems and comorbidity. A recent meta-analytic review of CBT supported by or delivered through information technology (IT) found no significant differences between face-to-face and computer-aided psychotherapy for anxiety and no differences between the system used (PC, palmtop, other) . However, many studies of computer-aided CBT have used small samples. It is also a concern that most studies of computer support in CBT lack details on therapist contacts, time spent on assessment and diagnosis, and sparse reports of interventions delivered by ordinary therapists . When introducing cell phone support in therapy in front-line settings, it is therefore important to be specific regarding what is compared, and to provide a large enough sample to be able to determine differences in effectiveness.
The role of the therapist
Another important issue in effectiveness evaluations of IT-supported therapy is non-adherence. In RCTs of computer-supported CBT, different methods have been used to deal with dropouts (e.g. including only completers) . Based on the evidence from the RCTs reported, it is tempting to meet the growing demand for CBT with computer-based therapy programs. However, although effective computer-based treatments have been tailored for specific and well-diagnosed patients, in practice there is a substantial comorbidity between anxiety and other mental illnesses, and the therapists variable in this context is still not well understood [21, 22]. Tailoring of Internet therapies according to diagnostic profiles may be possible; however, visitors to therapy web sites who have registered on their own initiative have also had larger dropout rates than that reported in RCTs .
Organizational and community context
To examine the effectiveness rather than the efficacy of CBT, it is important to include comorbid disorders and base the study on total populations. Reviews of RCTs of CBT for anxiety disorders have reported large pre- to post-treatment within-group effect sizes on measures of anxiety when supplied in therapist consultations in computer-supported sessions at practices [8, 9] and on the Internet [10, 11]. Yet, effective dissemination of evidence-based intervention to community stakeholders has been shown to be challenging . Facing the difficulties of closing the gap between research and practice, the UCLA/RAND NIMH Center has developed a framework to assess the relevant contextual factors, identify, develop and evaluate new strategies, provide useful formative feedback, develop capacity among stakeholders and to meaningfully generalize findings. Evidence on dissemination and implementation in an organizational and community context rests on process evaluation of contextual factors, stages of diffusion, and outcome of interventions . A successful outcome in one implementation is only anecdotal evidence, unless some evaluation of the context (structure and process) and adaptations made for the context are also documented.
The final protocol (NCT01205191 at clinicaltrials.gov) informed by the data analysis was developed with regard to the observed limitations of earlier studies of computer-supported CBT; e.g. the substantial dropout rates, partial neglect of therapist involvement, and lack of direct comparisons between face-to-face and IT-supported methods . The protocol is presented as it is implemented in a study at the Unga Vuxna (Young Adults) clinic in Östergötland County, Sweden.
Study inclusion criteria
The inclusion criteria are: (1) age 18 to 25 years and (2) scoring 7 or higher on the anxiety section of the Hospital Anxiety and Depression Scale (HAD) at initial evaluation. Clients are excluded if they: (1) display symptoms at the initial evaluation that suggest referral to a psychiatric specialist, e.g. active suicidal ideas or symptoms of a psychotic disorder, organic mental disorder or alcohol and/or drug dependence or (2) are unable to read, write or speak Swedish.
Clients receive whatever treatment their therapist prescribes. To replicate natural conditions, interventions received by clients are not constrained by the study. The interventions thus also include, for example, provision of practical/social support at the employment office, and further medical investigations.
Study outcome measures
The primary outcome measure is the Beck Anxiety Inventory [BAI]. The BAI  is a 21-item symptom checklist rated on a 4-point scale (0-3). The main secondary set of outcome measures describes patient adherence. Data on adherence are collected in 4 dimensions from the therapists after the last session; that is whether the client has
discontinued the CBT program by not showing up at sessions;
discontinued the CBT program by actively informing the therapist;
been discontinued from the treatment by the therapist; and
not fulfilled the therapeutic contract, e.g. by not completing homework agreed upon.
The remaining secondary outcome measures are HADS-A and -D, and the General Health Questionnaire [GHQ-12]. The outcome measures are administered on each of 3 occasions: pre-treatment, when the participants had completed their programme, and with follow-up at 6 months. The GHQ has demonstrated validity with young adults (17+ years) from the United Kingdom . The sensitivity and specificity for both HADS-A and HADS-D are similar to the sensitivity and specificity achieved by the GHQ and they perform well in assessing the symptom severity of anxiety disorders and depression in both somatic, psychiatric and primary care patients, and in the general population .
Power calculations were performed to determine the minimum sample size needed to fulfil the research aims. The calculations were based on change scores (pre- to post-treatment) between groups in 2 previous studies comparing CBT as usual with CBT supported by handheld computers  and CBT provided by stationary surgery-located computers . To detect a standard deviation difference of 0.3 in BAI change scores at 80% power and with 0.05 alpha, we determined that a total sample size close to 150 per stand (450 in total) was needed.
For each of the outcome measures, a series of mixed effects models will be considered. Such models are described in detail by Everitt and Pickles . Model adjustments will be used to analyse associations between pre- and post-randomization values of outcome (i.e. post-CBT and 6 months follow-up), treatment, drugs, and parallel depression. For dimensional outcomes (anxiety scores), ITT analyses will be conducted using all participants enrolled in the study, and the last available data point will be used as the post-treatment score. Uncontrolled effect sizes will be calculated for pre- to post-treatment change. For categorical outcomes (adherence), treatments will be compared using Pearson's chi-square test, Fisher's exact test, or logistic regression, as appropriate.
The study protocol was approved by the Regional Research Ethics Committee in Linköping in 2009 (Dnr. 111-09). Before submission to the committee the protocol was reviewed with regard to ethical issues related to the participants, particularly considering non-malfeasance, autonomy, and fidelity . Prior to invitation and randomization, clients displaying signs of severe psychiatric disease are referred to psychiatric specialists. The research will contribute to the development of the evidence base in the area, with potential future benefits for participants and society. All participation will be based on informed consent. The clients will be able to withdraw from the study at any time, while still being supplied routine CBT services. The integrity of the randomization procedure will be checked on a regular basis.
Health care services are increasingly moving out of the physician's office into IT-mediated treatment in the community [18, 31]. There is evidence that IT-supported therapy works in controlled settings . However, if evidence is to be translated from stringent RCTs to frontline clinical practice, some obstacles still remain before IT-supported therapy can become routine practice. Two particular differences between evidence from RCTs and frontline clinical settings that need to be investigated are their applicability to total populations and on clients with overt comorbidity. In addition, there are a host of organizational and community context issues that can only be investigated in real-world settings.
Different applications of cell phone support for in situ CBT have recently been reported . The present protocol can be used to study the effectiveness of cell phone-supported CBT for management of anxiety in different practice, organizational, and technical settings. We recognize the lessons learned from early innovative studies that had methodological shortcomings preventing them from transforming from evidence to practice . Although different types of software applications for diagnosis and treatment of anxiety syndromes are available off-the-shelf for stationary computers and the Internet, the availability of high-quality applications for portable computers and cell phones is scarce. To allow the CBT-cell-phone strand to be replicated in other studies, the design of the application used in the case study setting will be described in detail in forthcoming reports.
The study protocol complies with allocation concealment and completeness of follow-up data. However, because of the tangible nature of cell-phone supported therapy, blinding is not supported by the protocol. After having been allocated a client and a treatment in a concealed process, the therapist has total insight in the therapy and, thereby, any use of supportive technologies is not blinded.
This protocol for use in frontline clinical practice in which effectiveness, adherence, and the role of the therapists are analysed will provide evidence for what are truly valuable cell-phone-supported CBT treatments and guidance for the broader introduction of CBT in health services.
This research is supported by a grant from Länsförsäkringars Forskningsfond under contract P5/08. We are grateful to the reviewer for valuable comments on earlier drafts of the manuscript.
- WHO: The world health report: 2001 - mental health: new understanding, new hope. 2001, Geneva: World Health OrganizationGoogle Scholar
- Goisman RM, Warshaw MG, Keller MB: Psychosocial treatment prescriptions for generalized anxiety disorder, panic disorder, and social phobia, 1991-1996. Am J Psychiatry. 1999, 156: 1819-1821.PubMedGoogle Scholar
- Taylor CB, Chang VY: Issues in the dissemination of cognitive-behavior therapy. Nord J Psychiatry. 2008, 62 (Suppl 47): 37-44. 10.1080/08039480802315673.View ArticlePubMedGoogle Scholar
- UK Department of Health: Improving access to psychological therapies: national guideline for regional delivery. 2008, London: Department of HealthGoogle Scholar
- Wang PS, Demler O, Kessler RC: Adequacy of treatment for serious mental illness in the United States. Am J Public Health. 2002, 92: 92-98. 10.2105/AJPH.92.1.92.View ArticlePubMedPubMed CentralGoogle Scholar
- Mitte K: Meta-analysis of cognitive-behavioral treatments for generalized anxiety disorder: a comparison with pharmacotherapy. Psychol Bull. 2005, 131: 785-795. 10.1037/0033-2909.131.5.785.View ArticlePubMedGoogle Scholar
- Butler AC, Chapman JE, Forman EM, Beck AT: The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clin Psychol Rev. 2006, 26: 17-31. 10.1016/j.cpr.2005.07.003.View ArticlePubMedGoogle Scholar
- Proudfoot J, Goldberg D, Mann A, Everitt B, Marks I, Gray JA: Computerized, interactive, multimedia cognitive-behavioural program for anxiety and depression in general practice. Psychol Med. 2003, 33: 217-227. 10.1017/S0033291702007225.View ArticlePubMedGoogle Scholar
- Marks IM, Mataix-Cols D, Kenwright M, Cameron R, Hirsch S, Gega L: Pragmatic evaluation of computer-aided self-help for anxiety and depression. Br J Psychiatry. 2003, 183: 57-65. 10.1192/bjp.183.1.57.View ArticlePubMedGoogle Scholar
- Andersson G, Carlbring P, Holmstrom A, Sparthan E, Furmark T, Nilsson-Ihrfelt E, Buhrman M, Ekselius L: Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomized controlled trial. J Consult Clin Psychol. 2006, 74: 677-686. 10.1037/0022-006X.74.4.677.View ArticlePubMedGoogle Scholar
- Carlbring P, Bohman S, Brunt S, Buhrman M, Westling BE, Ekselius L, Andersson G: Remote treatment of panic disorder: a randomized trial of internet-based cognitive behavior therapy supplemented with telephone calls. Am J Psychiatry. 2006, 163: 2119-2125. 10.1176/appi.ajp.163.12.2119.View ArticlePubMedGoogle Scholar
- SoS Socialstyrelsen: Depressionssjukdom och ångestsyndrom: Vetenskapligt underlag för Nationella riktlinjer. 2009Google Scholar
- Kehle SM: The effectiveness of cognitive behavioral therapy for generalized anxiety disorder in a frontline service setting. Cogn Behav Ther. 2008, 37: 192-198. 10.1080/16506070802190262.View ArticlePubMedGoogle Scholar
- Kopta SM, Lueger RJ, Saunders SM, Howard KI: Individual psychotherapy outcome and process research: challenges leading to greater turmoil or a positive transition?. Annu Rev Psychol. 1999, 50: 441-469. 10.1146/annurev.psych.50.1.441.View ArticlePubMedGoogle Scholar
- Westen D, Novotny CM, Thompson-Brenner H: The empirical status of empirically supported psychotherapies: assumptions, findings, and reporting in controlled clinical trials. Psychol Bull. 2004, 130: 631-663. 10.1037/0033-2909.130.4.631.View ArticlePubMedGoogle Scholar
- Wade WA, Treat TA, Stuart GL: Transporting an empirically supported treatment for panic disorder to a service clinic setting: a benchmarking strategy. J Consult Clin Psychol. 1998, 66: 231-239. 10.1037/0022-006X.66.2.231.View ArticlePubMedGoogle Scholar
- Bang M, Timpka T, Eriksson H, Holm E, Nordin C: Mobile phone computing for in-situ cognitive behavioral therapy. Stud Health Technol Inform. 2007, 129: 1078-1082.PubMedGoogle Scholar
- Delbanco T, Berwick DM, Boufford JI, Edgman-Levitan S, Ollenschlager G, Plamping D, Rockefeller RG: Healthcare in a land called PeoplePower: nothing about me without me. Health Expect. 2001, 4: 144-150. 10.1046/j.1369-6513.2001.00145.x.View ArticlePubMedGoogle Scholar
- Jones J, Hunter D: Consensus methods for medical and health services research. BMJ. 1995, 311: 376-380.View ArticlePubMedPubMed CentralGoogle Scholar
- Cuijpers P, Marks I, Straten A, Cavanagh K, Gega L, Andersson G: Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn Behav Ther. 2009, 38: 66-82. 10.1080/16506070802694776.View ArticlePubMedGoogle Scholar
- Andersson G, Carlbring P, Berger T, Almlov J, Cuijpers P: What makes internet therapy work?. Cogn Behav Ther. 2009, 38 (S1): 55-60. 10.1080/16506070902916400.View ArticlePubMedGoogle Scholar
- Almlov J, Carlbring P, Berger T, Cuijpers P, Andersson G: Therapist factors in internet-delivered cognitive behavioural therapy for major depressive disorder. Cogn Behav Ther. 2009, 38: 247-254. 10.1080/16506070903116935.View ArticlePubMedGoogle Scholar
- Eysenbach G: The law of attrition. J Med Internet Res. 2005, 7: e11-10.2196/jmir.7.1.e11.View ArticlePubMedPubMed CentralGoogle Scholar
- Mendel P, Meredith LS, Schoenbaum M, Sherbourne CD, Wells KB: Interventions in organizational and community context: a framework for building evidence on dissemination and implementation in health services research. Adm Policy Ment Health. 2008, 35: 21-37. 10.1007/s10488-007-0144-9.View ArticlePubMedGoogle Scholar
- Beck AT, Steer RA: BAI, Beck Anxiety Inventory. 1990, San Antonio, TX: Psychological Corporation, Harcourt Brave JovanovichGoogle Scholar
- Tait RJ, Hulse GK, Robertson SI: A review of the validity of the General Health Questionnaire in adolescent populations. Aust N Z J Psychiatry. 2002, 36: 550-557. 10.1046/j.1440-1614.2002.01028.x.View ArticlePubMedGoogle Scholar
- Bjelland I, Dahl AA, Haug TT, Neckelmann D: The validity of the Hospital Anxiety and Depression Scale. An updated literature review. J Psychosom Res. 2002, 52: 69-77. 10.1016/S0022-3999(01)00296-3.View ArticlePubMedGoogle Scholar
- Kenardy JA, Dow MG, Johnston DW, Newman MG, Thomson A, Taylor CB: A comparison of delivery methods of cognitive-behavioral therapy for panic disorder: an international multicenter trial. J Consult Clin Psychol. 2003, 71: 1068-1075. 10.1037/0022-006X.71.6.1068.View ArticlePubMedGoogle Scholar
- Everitt BS, Pickles A: Statistical aspects of the design and analysis of clinical trials. 1999, London: Imperial College PressGoogle Scholar
- Heppner PP, Wampold BE, Kivligan DM: Research design in counselling. 2008, Belmont, CA: Thomson, 3Google Scholar
- Delbanco T, Sands DZ: Electrons in flight - e-mail between doctors and patients. N Engl J Med. 2004, 350: 1705-1707. 10.1056/NEJMp038209.View ArticlePubMedGoogle Scholar
- Morris ME, Kathawala Q, Leen TK, Gorenstein EE, Guilak F, Labhard M, Deleeuw W: Mobile therapy: case study evaluations of a cell phone application for emotional self-awareness. J Med Internet Res. 2010, 12 (2): e10-10.2196/jmir.1371.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/11/3/prepub