To explore the strengths and weaknesses of conventional biomedical research strategies and methods as applied to complementary and alternative medicine (CAM), and to suggest a new research framework for assessing these treatment modalities.
There appears to be a gap between published studies showing little or no efficacy of CAM, and reports of substantial clinical benefit from patients and CAM practitioners. This "gap" might be partially due to the current focus on placebo-controlled randomized trials, which are appropriately designed to answer questions about the efficacy and safety of pharmaceutical agents. In an attempt to fit this assessment strategy, complex CAM treatment approaches have been dissected into standardized and often simplified treatment methods, and outcomes have been limited.
Unlike conventional medicine, CAM has no regulatory or financial gatekeeper controlling their therapeutic "agents" before they are marketed. Treatments may thus be in widespread use before researchers know of their existence. In addition, the treatments are often provided as an integrated 'whole system' of care, without careful consideration of the safety issue.
We propose a five-phase strategy for assessing CAM built on the acknowledgement of the inherent, unique aspects of CAM treatments and their regulatory status in most Western countries. These phases comprise:
Context, paradigms, philosophical understanding and utilization
Using the proposed strategy will generate evidence relevant to clinical practice, while acknowledging the absence of regulatory and financial gatekeepers for CAM. It will also emphasize the important but subtle differences between CAM and conventional medical practice.
The use of complementary and alternative medicine (CAM) has increased considerably in Western industrialized nations over the last 25 years. In the USA, the expenditure is approximately $30 billion per annum, surpassing current out-of-pocket expenditures for conventional treatments by primary care physicians [1, 2]. CAM treatment modalities include a variety of approaches (i.e., acupuncture, homeopathy, herbal medicine, massage, reflexology, Reiki healing etc.), many of them based on theories that differ markedly from conventional Western biomedicine. The assessment and management of illness often includes a more detailed interest in patients' wellbeing, as well as recommendations concerning lifestyle and their quality of life as a whole .
Patients are selective in their choice of CAM treatments , usually abstaining from exclusive use of CAM in acute life-threatening conditions. Rather, CAM is mainly used in addition to conventional care for chronic and some acute health conditions, for disease prevention and for maintaining wellness. For example, more than half of all breast cancer patients use some form of CAM complementary to conventional medicine, most often with a palliative intent [5, 6].
Many physicians do not understand why large segments of their patients use CAM when research generally has failed to provide decisive evidence of efficacy. Patients seem to make these treatment choices based on the qualities of the provider, desire for "individualized" treatments, and their perception of overall effectiveness rather than efficacy [7–11].
There is apparently a "gap" between the results of randomized controlled trials (RCT) showing little or no effect and the widespread use and reports of beneficial outcomes of CAM treatment [12, 13]. If we were to assume that patients are not completely misguided, then we would need to look closely at the research strategies utilized in the CAM field and try to understand the reasons for the gap.
The purpose of this paper is therefore to
Explore the strengths and weaknesses of conventional biomedical research strategies and methods as applied to CAM
Suggest a new research framework for assessing these treatment modalities.
Gatekeeping and regulating the use of CAM interventions
A majority of CAM research to date has used the research strategy employed and developed by clinical pharmacologists to document, in a prescribed sequential pattern, the quality, dose, safety, efficacy, and eventual effectiveness of a drug prior to its general release. In this model, governmental regulatory offices (e.g., USA: FDA, EU: EMEA) act as gatekeepers. Health insurers, including governmental single-payer funders, only reimburse for drugs meeting the appropriate criteria. An important principle in this research-regulatory-utilization model is that research determines which drugs are approved for generalized clinical use and are paid for by health insurers. There is also a growing network of international Health Technology Assessment Boards that seek consensus about the use of medical procedures, and some suggest this approach should be extended to the area of diagnostic tests .
Previously, CAM researchers have largely assumed that these same pharmacological research methods and regulatory-reimbursement models can be followed in the evaluation of CAM. Much to the frustration of many clinical researchers, the availability of CAM treatments which are affordable with little or no reimbursement, however, does not seem to be amenable to the same rules as those we seek to apply to conventional medicine. Even if studies show that a CAM treatment has no effect, it does not necessarily disappear from the marketplace. We suggest that this is because the CAM market has had no statutory body (gatekeeper) that ensured the quality, safety, efficacy and effectiveness of CAM treatments before they appear on the "market". Further, since few health insurers reimburse for CAM treatments, there are few financial gatekeepers. Millions of patients in developed countries have experienced the effect of CAM treatments provided in contrast to, or more likely in addition to, the care they have received in conventional medicine [15–19]. The situation is thereby characterized by widespread patient experience of treatment outcomes combined with little research and no regulatory or financial gate-keeping activity.
Why is a different research strategy needed?
In conventional medicine we rarely question the efficacy or effectiveness of the overall therapeutic approach to the treatment of sick people. We take for granted that seeking advice from a health care professional is a good way of dealing with diverse health problems. Research in conventional medicine therefore focuses on choosing the best tools for health professionals to use. These tools include drugs, diagnostic methods, and surgical procedures, and employ a methodology often resulting in a one-size-fits-all therapeutic prescription. Conventional research may, however, have overlooked that the clinical effect of most therapies are overestimated when studied under optimal circumstances on susceptible, cooperative patients. Despite this, the randomized controlled trial is an important method when making decisions about tools to include in the toolbox of conventional medicine.
Because "conventional" researchers have done most of the CAM research, it is not surprising that CAM research has traveled down the same path as conventional medicine. It has tested the specific efficacy of what conventional researchers believe to be the active components of a therapy, often discounting synergistic effects. In addition to providing a predominantly individualistic treatment approach, many CAM therapists hold that CAM treatments cannot be split up into parts that can be investigated separately. They argue that the total effect adds up to more than the sum of its parts.
If CAM is to be evaluated comprehensively, one needs to extend the research focus to all aspects of the treatment approach . To study only the specific effect of needling in acupuncture isolated from other interventions initiated by the acupuncturist, or the effect of a single, isolated homeopathic remedy separated from other aspects of homeopathic practice, is to neglect other potentially important components of these interventions [21–25]. Furthermore, clinical research teams should only venture into this area with a thorough contextual and philosophical understanding of the CAM treatment paradigm and its clinical use. Plunging into studies of efficacy that involve isolated detailed components of a treatment approach without thoroughly understanding its context is destined to failure and irrelevance no matter what the results show. These issues point to the need for a different, and more complex, research strategy for the CAM field.
A suggested research strategy
The most important unique characteristics of CAM are the absence of gatekeepers and the complexity of individually tailored treatments. Our proposed strategy does not contain new methodological elements, but organizes existing elements in a way that is tailored to pragmatic clinical practice (Figure 1). The strategy is not meant as a strictly chronological sequence defining when specific research phases should occur, but as a framework to guide CAM research, illustrating the necessary building blocks required for a rigorous evidence base.
Although we have developed this strategy within a CAM context, it is appropriate for any treatment approach that has developed in and from clinical practice, and whose specific treatment tools are unregulated. In conventional medicine this might include nursing, health psychology, counseling and some aspects of general practice. These areas struggle, as does CAM, to establish a rigorous research framework around which their research efforts can be organized [26–28].
Phase1: Context, paradigms, philosophical understanding and utilization (What is going on?)
The cornerstone of the suggested research strategy is to understand the process and assumptions within a particular therapy, often using an inductive research approach [29–34]. Researchers need to understand what the treatment procedure is, how many variations there are, what philosophical foundations underlie it, its ideas about health and disease, its contextual framework and key treatment components. They also need to know how many and what segments of the patient population use it, and for what conditions. Many research questions can be asked within phase I, including perceived benefit of the treatment, cost of treatment, qualifications of the providers etc.
Phase 2: Safety status (Is it safe?)
Safety issues are important in the treatment of any illness, but in the area of CAM they cannot be emphasized strongly enough. The inherent risks of the diseases or illnesses for which patients seek CAM are generally low. The treatments given should therefore carry a low risk of adverse effects. CAM treatments have often been claimed to be without risk, but adverse effects in CAM are more than occasional case reports. . The methods of choice to study safety would be similar to the detection of adverse events associated with pharmaceutical treatments. The field of acupuncture has provided a thorough and rigorous risk assessment [36–39] based on these principles and the other fields within CAM need to follow suit [40–42].
Phase 3: Comparative effectiveness (What is the system effectiveness?)
Patients are seeking CAM as a treatment system. Research needs to examine the outcomes of these treatments both in combination with, and as alternative, to conventional care. Thus randomized, controlled pragmatic trials are needed [43, 44], wherein the specifics of the system are not disassembled, but the system under study is allowed to function as it is clinically practiced, including the urgently needed evaluation of cost-effectiveness . The pragmatic trial design has been used in the study of acupuncture treatment of chronic pain , and the methodology is also widely used in conventional medicine, for example in dietary intervention studies such as the Women's Health Initiative . In these trials, patients are randomly assigned to treatment alternatives which may include alternative viable whole systems, conventional treatment or no treatment. The trials may also evaluate whole system interventions that are implemented under protocols that specify individualized treatments based on specific patient characteristics, following the patterns of care found in the community. While blinding of the treatment providers and patients/subjects with regard to treatment allocation is usually not feasible, blinding of the outcomes evaluators can ensure an unbiased comparison of the outcome assessment.
In conventional medicine, a treatment's efficacy is often determined before assessment of its effectiveness. It is considered unethical to include non-efficacious treatments in the real world treatment of patients. Patients are, however, already using CAM treatments and thus effectiveness studies can be used to guide decisions about the necessity of studying the efficacy of specific components. Researchers have voiced this view in several countries over the last few years [20, 48, 49]. It was even emphasized in the much-criticized recent meta-analysis of homeopathy : "Clearly, rather than doing further placebo-controlled trials of homoeopathy, future research efforts should focus on the nature of context effects and on the place of homoeopathy in health-care systems".
Included in this phase, and bordering on the next phase would be studies to evaluate how limited a whole package of care can be while still retaining its overall effectiveness ("Occam's razor"). There are many components within a CAM treatment approach, but are there any that we can eliminate and at the same time retain or improve overall treatment effectiveness?
Phase 4: Component efficacy (What is the efficacy of a specific component of the therapy?)
This is the area that has received most attention and research money to date, and while it is important, it is not the starting point in our model. The methods of choice are often double-blind randomized controlled trials. The well-established documentation of acupuncture/acupressure stimulation of one acupuncture point in the treatment of chemotherapy-induced nausea/vomiting is such an example . It is, however, important to recognize that results from such research cannot be used to document or disprove the effectiveness of a "whole system" treatment.
Phase 5: Biological mechanisms (How can treatment outcomes be explained biologically?)
We want, and need, to understand the pathways and mechanisms through which treatments exercise their influence . This has been and is being explored for acupuncture as an anti-emetic . We must however realize that treatment outcomes, both at the system and component level, can be documented before the biologic mechanisms are understood, as has been the case for several non-surgical conventional therapies. The most prominent example from conventional medicine is probably aspirin. The anti-inflammatory and pain-killing properties of aspirin were discovered long before it became known that aspirin influences prostaglandin synthesis.
CAM is not simply a new array of therapeutic tools that need to be evaluated; it presents other ways to think about disease and therapeutics, and consequently new ideas about how research should be strategically developed. In this article we have suggested two ways of taking this forward. First, the absence of statutory and financial gatekeepers for CAM presents several issues that need to be considered closely. Secondly, the structure of CAM research should be different, in subtle but important ways. We have provided some suggestions as to how this alternative research strategy could be structured keeping in mind that the ultimate goal for all approaches to treatment is to provide effective medical interventions at reasonable cost and without harm.
National Research Center in Complementary and Alternative Medicine, Faculty of Medicine, University of Tromsø
Clinical Research Center, University Hospital of North Norway
School of Social Sciences & Samueli Institute, University of Northampton
Department of Family & Community Medicine, University of Arizona
Department of Health Sciences, University of York
Complementary Medicine Research Unit, Primary Medical Care, Aldermoor Health Centre
Program in Integrative Medicine, University of Arizona
Center for Studies of Complementary Medicine, Department of Nursing and the division of International Health (IHCAR), Department of Public Health Sciences, Karolinska Institutet
Leslie Dan Faculty of Pharmacy, University of Toronto
Eisenberg DM, Kessler RC, Foster C, Norlock FE, Calkins DR, Delbanco TL: Unconventional Medicine in the United-States – Prevalence, Costs, and Patterns of Use.New England Journal of Medicine 1993, 328:246–252.View ArticlePubMed
Eisenberg DM, Davis RB, Ettner SL, Appel S, Wilkey S, van Rompay M, Kessler RC: Trends in alternative medicine use in the United States, 1990–1997 – Results of a follow-up national survey.Jama-Journal of the American Medical Association 1998, 280:1569–1575.View Article
Steinsbekk A, Launso L: Empowering the cancer patient or controlling the tumor? A qualitative study of how cancer patients experience consultations with complementary and alternative medicine practitioners and physicians, respectively.Integr Cancer Ther 2005, 4:195–200.View ArticlePubMed
Caspi O, Koithan M, Criddle MW: Alternative medicine or "alternative" patients: a qualitative study of patient-oriented decision-making processes with respect to complementary and alternative medicine.Med Decis Making 2004, 24:64–79.View ArticlePubMed
Von RE, Pampallona S, Van WB, Cerny T, Hurny C, Bernhard J, Helwig S, Heusser P: Attitudes and Beliefs towards Disease and Treatment in Patients with Advanced Cancer Using Anthroposophical Medicine.Onkologie 2000, 23:558–563.View Article
Weis J, Bartsch HH, Hennies F, Rietschel M, Heim M, Adam G, Gartner U, Ammon A: Complementary medicine in cancer patients: Demand, patients' attitudes and psychological beliefs.Onkologie 1998, 21:144–149.View Article
Wilkinson S, Gomella LG, Smith JA, Brawer MK, Dawson NA, Wajsman Z, Dai L, Chodak GW: Attitudes and use of complementary medicine in men with prostate cancer.J Urol 2002, 168:2505–2509.View ArticlePubMed
Schonekaes K, Micke O, Mucke R, Buntzel J, Glatzel M, Bruns F, Kisters K: [Use of complementary/alternative therapy methods by patients with breast cancer].Forsch Komplementarmed Klass Naturheilkd 2003, 10:304–308.View ArticlePubMed
Astin JA: Why patients use alternative medicine: results of a national study.JAMA 1998, 279:1548–1553.View ArticlePubMed
Boon H, Brown JB, Gavin A, Kennard MA, Stewart M: Breast cancer survivors' perceptions of complementary/alternative medicine (CAM): making the decision to use or not to use.Qual Health Res 1999, 9:639–653.View ArticlePubMed
Boon H, Brown JB, Gavin A, Westlake K: Men with prostate cancer: making decisions about complementary/alternative medicine.Med Decis Making 2003, 23:471–479.View ArticlePubMed
Fønnebø V, Launsø L: Looking for new knowledge in the field of curing and healing.Focus Altern Complement Ther 2005, 10:13–14.
Launsø L, Gannik DE: The need for revision of medical research designs.Disease knowledge and society(Edited by: Gannik DE and Launsø L). Copenhagen, Forlaget Samfundslitteratur 2000.
Guthlin C, Lange O, Walach H: Measuring the effects of acupuncture and homoeopathy in general practice: an uncontrolled prospective documentation approach.BMC Public Health 2004, 4:6.View ArticlePubMed
Witt CM, Ludtke R, Baur R, Willich SN: Homeopathic medical practice: long-term results of a cohort study with 3981 patients.BMC Public Health 2005, 5:115.View ArticlePubMed
Hamre HJ, Becker-Witt C, Glockmann A, Ziegler R, Willich SN, Kiene H: Anthroposophic therapies in chronic disease: the Anthroposophic Medicine Outcomes Study (AMOS).Eur J Med Res 2004, 9:351–360.PubMed
Steinsbekk A, Ludtke R: Patients' assessments of the effectiveness of homeopathic care in Norway: a prospective observational multicentre outcome study.Homeopathy 2005, 94:10–16.View ArticlePubMed
Norheim AJ, Fonnebo V: A survey of acupuncture patients: results from a questionnaire among a random sample in the general population in Norway.Complement Ther Med 2000, 8:187–192.View ArticlePubMed
Ritenbaugh C, Verhoef M, Fleishman S, Boon H, Leis A: Whole systems research: a discipline for studying complementary and alternative medicine.Altern Ther Health Med 2003, 9:32–36.PubMed
Walach H, Jonas WB, Lewith G: The role of outcomes research in evaluating complementary and alternative medicine.Clinical Research in Complementary Therapies: Principles, Problems, and Solutions(Edited by: Lewith G, Jonas WB and Walach H). London, Churchill-Livingston 2002, 29–45.View Article
Lewith G, Walach H, Jonas WB: Balanced research strategies for complementary and alternative medicine.Clinical Research in Complementary Therapies: Principles, Problems, and Solutions London, Churchill Livingston 2002, 3–27.
Jonas WB: Building an evidence house: challenges and solutions to research in complementary and alternative medicine.Forsch Komplementarmed Klass Naturheilkd 2005, 12:159–167.View ArticlePubMed
Wittman WW, Walach H: Evaluating complementary medicine: Lessons to be learned from evaluation research.Clinical Research in Complementary Therapies: Principles, Problems, and Solutions London, Churchill Livingston 2002, 93–108.View Article
Paterson C, Dieppe P: Characteristic and incidental (placebo) effects in complex interventions such as acupuncture.BMJ 2005, 330:1202–1205.View ArticlePubMed
Science and Pseudoscience in psychology (Edited by: Lilienfeld SO, Lynn SJ and Lohr JM). New York, The Guilford Press 2003.
Baskin TW, Tierney SC, Minami T, Wampold BE: Establishing specificity in psychotherapy: a meta-analysis of structural equivalence of placebo controls.J Consult Clin Psychol 2003, 71:973–979.View ArticlePubMed
Horvath P: Placebos and common factors in two decades of psychotherapy research.Psychol Bull 1988, 104:214–225.View ArticlePubMed
Thomas KJ, Nicholl JP, Coleman P: Use and expenditure on complementary medicine in England: a population based survey.Complement Ther Med 2001, 9:2–11.View ArticlePubMed
Thomas KJ, Carr J, Westlake L, Williams BT: Use of non-orthodox and conventional health care in Great Britain.BMJ 1991, 302:207–210.View ArticlePubMed
Hanssen B, Grimsgaard S, Launso L, Fonnebo V, Falkenberg T, Rasmussen NK: Use of complementary and alternative medicine in the Scandinavian countries.Scand J Prim Health Care 2005, 23:57–62.View ArticlePubMed
Launso L, Brendstrup E, Arnberg S: An exploratory study of reflexological treatment for headache.Altern Ther Health Med 1999, 5:57–65.PubMed
Launso L, Rieper J: General practitioners and classical homeopaths treatment models for asthma and allergy.Homeopathy 2005, 94:17–25.View ArticlePubMed
Launso L: Use of alternative treatments in Denmark: patterns of use and patients' experience with treatment effects.Altern Ther Health Med 2000, 6:102–107.PubMed
Norheim AJ, Fønnebø V: Acupuncture adverse effects are more than occasional case reports.Complement Ther Med 1996, 4:14–20.View Article
Norheim AJ: Adverse effects of acupuncture: a study of the literature for the years 1981–1994.J Altern Complement Med 1996, 2:291–297.View ArticlePubMed
MacPherson H, Thomas K, Walters S, Fitter M: The York acupuncture safety study: prospective survey of 34 000 treatments by traditional acupuncturists.BMJ 2001, 323:486–487.View ArticlePubMed
MacPherson H, Scullion A, Thomas KJ, Walters S: Patient reports of adverse events associated with acupuncture treatment: a prospective national survey.Qual Saf Health Care 2004, 13:349–355.View ArticlePubMed
MacPherson H, Liu B: The safety of Chinese herbal medicine: a pilot study for a national survey.J Altern Complement Med 2005, 11:617–626.View ArticlePubMed
Endrizzi C, Rossi E, Crudeli L, Garibaldi D: Harm in homeopathy: aggravations, adverse drug events or medication errors?Homeopathy 2005, 94:233–240.View ArticlePubMed
Thompson E, Barron S, Spence D: A preliminary audit investigating remedy reactions including adverse events in routine homeopathic practice.Homeopathy 2004, 93:203–209.View ArticlePubMed
MacPherson H: Pragmatic clinical trials.Complement Ther Med 2004, 12:136–140.View ArticlePubMed
Hammerschlag R: Methodological and ethical issues in clinical trials of acupuncture.J Altern Complement Med 1998, 4:159–171.View ArticlePubMed
Thomas KJ, MacPherson H, Ratcliffe J, Thorpe L, Brazier J, Campbell M, Fitter M, Roman M, Walters S, Nicholl JP: Longer term clinical and economic benefits of offering acupuncture care to patients with chronic low back pain.Health Technol Assess 2005, 9:iii-x, 1.
Vickers AJ, Rees RW, Zollman CE, McCarney R, Smith CM, Ellis N, Fisher P, Van HR: Acupuncture for chronic headache in primary care: large, pragmatic, randomised trial.BMJ 2004, 328:744.View ArticlePubMed
Beresford SA, Johnson KC, Ritenbaugh C, Lasser NL, Snetselaar LG, Black HR, Anderson GL, Assaf AR, Bassford T, Bowen D, Brunner RL, Brzyski RG, Caan B, Chlebowski RT, Gass M, Harrigan RC, Hays J, Heber D, Heiss G, Hendrix SL, Howard BV, Hsia J, Hubbell FA, Jackson RD, Kotchen JM, Kuller LH, LaCroix AZ, Lane DS, Langer RD, Lewis CE, Manson JE, Margolis KL, Mossavar-Rahmani Y, Ockene JK, Parker LM, Perri MG, Phillips L, Prentice RL, Robbins J, Rossouw JE, Sarto GE, Stefanick ML, Van HL, Vitolins MZ, Wactawski-Wende J, Wallace RB, Whitlock E: Low-fat dietary pattern and risk of colorectal cancer: the Women's Health Initiative Randomized Controlled Dietary Modification Trial.JAMA 2006, 295:643–654.View ArticlePubMed
Verhoef MJ, Lewith G, Ritenbaugh C, Boon H, Fleishman S, Leis A: Complementary and alternative medicine whole systems research: beyond identification of inadequacies of the RCT.Complement Ther Med 2005, 13:206–212.View ArticlePubMed
Walach H: Zirkulär statt hierarchisch: Methodische Prinzipien bei der Evaluation der therapeutischen Effekte von Komplementärmedizin und anderer komplexer Massnahmen.Informatik, Biometrie und Epidemiologie in Medizin und Biologie 2004, 35:229–242.
Shang A, Huwiler-Muntener K, Nartey L, Juni P, Dorig S, Sterne JA, Pewsner D, Egger M: Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy.Lancet 2005, 366:726–732.View ArticlePubMed
Ezzo J, Vickers A, Richardson MA, Allen C, Dibble SL, Issell B, Lao L, Pearl M, Ramirez G, Roscoe JA, Shen J, Shivnan J, Streitberger K, Treish I, Zhang G: Acupuncture-point stimulation for chemotherapy-induced nausea and vomiting.J Clin Oncol 2005, 23:7188–7198.View ArticlePubMed
Lewith GT, White PJ, Pariente J: Investigating acupuncture using brain imaging techniques: the current state of play.Evid Based Complement Alternat Med 2005, 2:315–319.View ArticlePubMed
Tatewaki M, Strickland C, Fukuda H, Tsuchida D, Hoshino E, Pappas TN, Takahashi T: Effects of acupuncture on vasopressin-induced emesis in conscious dogs.Am J Physiol Regul Integr Comp Physiol 2005, 288:R401–R408.PubMed
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.