Are we drawing the right conclusions from randomised placebo-controlled trials? A post-hoc analysis of data from a randomised controlled trial
© van Die et al; licensee BioMed Central Ltd. 2009
Received: 03 December 2008
Accepted: 23 June 2009
Published: 23 June 2009
Assumptions underlying placebo controlled trials include that the placebo effect impacts on all study arms equally, and that treatment effects are additional to the placebo effect. However, these assumptions have recently been challenged, and different mechanisms may potentially be operating in the placebo and treatment arms. The objective of the current study was to explore the nature of placebo versus pharmacological effects by comparing predictors of the placebo response with predictors of the treatment response in a randomised, placebo-controlled trial of a phytotherapeutic combination for the treatment of menopausal symptoms. A substantial placebo response was observed but no significant difference in efficacy between the two arms.
A post hoc analysis was conducted on data from 93 participants who completed this previously published study. Variables at baseline were investigated as potential predictors of the response on any of the endpoints of flushing, overall menopausal symptoms and depression. Focused tests were conducted using hierarchical linear regression analyses. Based on these findings, analyses were conducted for both groups separately. These findings are discussed in relation to existing literature on placebo effects.
Distinct differences in predictors were observed between the placebo and active groups. A significant difference was found for study entry anxiety, and Greene Climacteric Scale (GCS) scores, on all three endpoints. Attitude to menopause was found to differ significantly between the two groups for GCS scores. Examination of the individual arms found anxiety at study entry to predict placebo response on all three outcome measures individually. In contrast, low anxiety was significantly associated with improvement in the active treatment group. None of the variables found to predict the placebo response was relevant to the treatment arm.
This study was a post hoc analysis of predictors of the placebo versus treatment response. Whilst this study does not explore neurobiological mechanisms, these observations are consistent with the hypotheses that 'drug' effects and placebo effects are not necessarily additive, and that mutually exclusive mechanisms may be operating in the two arms. The need for more research in the area of mechanisms and mediators of placebo versus active responses is supported.
International Clinical Trials Registry ISRCTN98972974.
The placebo-controlled trial is considered the gold standard among clinical research designs. The challenge of rigorous scientific research is to accurately determine the specific effect of an intervention over and above the placebo effect, (also referred to as 'non-specific effects', or 'context effects'). Failure to do so may result in the rejection of the intervention as ineffective as a potential treatment, as any benefits are ascribed to a placebo effect. We question this approach and suggest that inappropriate rejection of potentially viable treatments may be occurring.
One obvious conclusion from this observation is that antidepressant medication does, in fact, exert a very small pharmacological effect. Another possible explanation that has been proposed is that different neurobiological mechanisms may be operating in the two arms. The placebo may induce effects via psychological mechanisms only In the absence of a pharmacological effect, while the active treatment works through pharmacological mechanisms alone . Some support for this hypothesis is derived from brain-imaging studies of depressed subjects, showing that placebo and active treatments induce quite different changes in brain function, despite exerting similar benefits [6–8]. Similarly, neurophysiological research on analgesia has suggested that expectation pathways, rather than pain pathways, may be stimulated by placebo treatment . Expectation of reward has been shown to be at least partly mediated by the dopaminergic system [10–13], stimulation of which may be activated by the brain opioid system [9, 14, 15]. There is evidence that both endogenous opioids [16, 17] and placebo-induced dopamine release may be relevant to the placebo effect [18–20]. Participant-related factors identified that may be responsible for the effects produced by placebos include Pavlovian conditioning resulting from prior exposure to the therapeutic intervention, and the expectation of reward (clinical benefit, in this case) .
In this setting, the current study analysed data from a previously published double-blind, placebo-controlled, RCT that had found no significant effect over placebo on any of the endpoints  for the study treatment, which was therefore concluded to exert no more than placebo effects. A comparison was made between predictors of the response in the placebo arm and predictors of the response to the active treatment. It was hypothesised that, if the additivitiy assumption is correct, then the same variables would predict the response in both groups.
This study was a post hoc analysis of data from an investigation of 93 participants who completed a randomised, placebo-controlled, double-blind trial. We have previously published the outcome data on efficacy of the therapy  and predictors of the placebo response in the placebo group only . Here we extend this evaluation to examine whether these predictors are also relevant to the treatment arm, and include data from all study participants. The original RCT had investigated the effect of a phytotherapeutic combination, consisting of the herbs Hypericum perforatum and Vitex agnus-castus, for menopausal symptoms in late-perimenopausal and postmenopausal women . Following entry to the study, a two-week non-treatment run-in preceded the 16-week treatment phase. Endpoints included flushing, overall menopausal symptoms measured on the Greene climacteric scale (GCS) and depressive symptoms measured on the Hamilton Depression Inventory (HDI), both well-validated widely available tools. The trial was approved by the Human Research Ethics Committee at Royal Melbourne Institute of Technology University.
As previously described , two Vitex agnus-castus tablets or matching placebos were administered daily, in addition to three Hypericum perforatum tablets or matching placebos. The placebos were identical to the herbal tablets in size, colour, coating, weight and packaging. Placebo tablets comprised the excipients used in the active tablets; these were cellulose, modified starch, magnesium stearate and calcium hydrogen-phosphate. The daily dosage of the herbs was 1,000 mg Vitex agnus-castus, and 5,400 mg Hypericum perforatum. All tablets were manufactured under the Code of Good Manufacturing Practice by MediHerb Australia Pty Ltd.
Data were analysed using Statistical Package for Social Science (SPSS) Version 16 with the assistance of a biostatistician. Data were analysed in two ways:
Firstly, focused tests of the difference between betas were conducted for each of the three individual endpoints. To do this, a series of hierarchical linear regression analyses were conducted using grouping as a dichotomous variable. An interaction variable was created for grouping × predictor for each potential predictor variable. The interaction of grouping and predictor was examined. Secondly, independent variables were assessed individually for their ability to predict the response in each arm on the three separate outcome measures. In each analysis, hierarchical regression was conducted in order to control for the baseline scores for the relevant outcome measures.
Response was defined as change in a favourable direction, that is, decrease in severity of symptoms. Because total GCS scores and GCS anxiety subscale scores were not independent, these were not entered simultaneously into a multiple regression analysis.
Differences between groups for predictors of week 16 endpoint scores, after controlling for baseline scores
Interaction of Group and Predictor
Greene Climacteric Scale
GCS score, study entry
GCS anxiety, study entry
Attitude to menopause
Predictors of week 16 endpoint scores for individual arms, after controlling for baseline scores
Active Treatment Group
Age at trial start
Previous Herb Med effective
Attitude to menopause
GCS score, study entry
GCS anxiety, study entry
Change in GCS scores during run-in
For the individual arms, predictors of response identified, after controlling for baseline scores, were as follows.
Anxiety at study entry
Anxiety at study entry significantly predicted placebo response on all the endpoints individually, with higher anxiety at entry associated with lower scores at end of treatment phase: flushes R 2 = 0.33, Std. β = -0.28, p = 0.03; GCS R 2 = 0.24, Std. β = -0.29, p = 0.04; HDI-17 scale R 2 = 0.34, Std. β = -0.66, p < 0.001. For the active treatment group, however, anxiety at study entry predicted lack of response for flushing, R 2 = 0.45, Std. β = 0.28 p = 0.02, and depression, measured on the HDI-17. R 2 = 0.28, Std. β = 0.31, p = 0.03.
None of the other predictors of the placebo response was relevant to the response in the active treatment arm (see Table 2).
In the active treatment group, positive attitude to menopause predicted response on GCS, R 2 = 0.34, Std. β = -0.27, p = 0.036.
Baseline severity of scores
For the placebo group, Pearson's bivariate correlations revealed positive correlations between baseline scores and subsequent percentage improvement during the treatment phase for total GCS scores and anxiety subscale scores. No relationship between severity of scores and subsequent response was found for any of the endpoints for the active treatment group.
In the current study, there was a significant interaction between predictors of the response to placebo and the study intervention. Anxiety at study entry and overall menopausal symptoms at study entry (GCS scores) differed significantly between the two arms as predictors of the response on all three endpoints of flushing, overall menopausal symptoms and depression. Attitude to menopause differed significantly in its predictive ability between the two groups for the response on GCS scores. In terms of the individual arms, there were distinct differences in predictors of outcome observed between the placebo and active groups. Anxiety at study entry predicted placebo response for all the endpoints. In contrast, for flushing and depression in the treatment arm, study entry anxiety significantly predicted a lack of response to treatment, and had no effect on the Greene Climacteric scale scores. None of the other variables that predicted the placebo response was relevant to the treatment response. Improvement during non-treatment run-in predicted subsequent improvement during the treatment phase on GCS and HDI-17 depression scores for the placebo arm. This trend was not mirrored in the active treatment arm. For depression scores, older age at study entry predicted placebo response, as did prior positive experience with phytotherapy. However neither of these variables significantly impacted on outcomes in the active treatment arm. For the Greene Climacteric scale, baseline severity of symptoms was positively correlated with percentage improvement across the treatment phase in the placebo group, but not in the active treatment group.
Previous researchers of a range of other conditions have compared predictors of the responses in placebo and active arms within the same study where effects between the two arms differed [24–27]. Severity of symptoms at baseline has been found to differentially predict the placebo and treatment responses, with more severe depression being less responsive to placebo but more responsive to the pharmacological intervention . Another study on acute bipolar manic episodes found symptom severity, age, number of previous hospitalisations to similarly predict the responses in both arms . With regard to change in symptom severity during run-in, significant worsening of symptoms was associated with subsequent placebo response, but not "drug response" in an analysis of data from a functional dyspepsia study . This contrasts with observations from the current study that improvement during run-in predicted subsequent response to placebo, but not to active treatment. However, to our knowledge, no previous studies have examined data from a RCT where superiority of 'active' treatment over placebo was not established to test the hypothesis that the predictors would be similar in the two arms.
In this study where efficacy of active and placebo were equivalent, the implications of the finding that the predictors of placebo response did not predict the treatment response are intriguing. As mentioned above, it has previously been suggested that the assumption of additivity of effects that underlies the practice of using placebos may not be a logical necessity . It is possible that psychological mechanisms may operate in the placebo arm only in the absence of pharmacological effects, whereas effective interventions activate pharmacological mechanisms to the exclusion of psychological mechanisms . Although it is generally accepted that there is a placebo component in the response to the active treatment when participants are blinded, the hypothesis of non-additivity implies that the pharmacological effects of an active intervention could override the psychologically-activated placebo component completely or partially. Essentially, the trial participants would experience either placebo or physiological intervention effects, but not both. If shown to be correct, this would invalidate the assumption that intervention effects are additive to placebo effects.
To our knowledge, no evidence exists from neurobiological studies of differential mechanisms operating in relation to menopausal symptoms, although there is some support for this phenomenon in relation to depression [6–8]. As depression, measured on the Hamilton Depression Inventory and the Greene Climacteric subscale, was one of endpoints of the current study, different mechanisms operating in the two arms in the current study cannot entirely be ruled out.
It is interesting to note that higher anxiety at study entry was a significant predictor of the placebo response, but predicted lack of response to active treatment. This supports the proposition that psychological factors are relevant to the placebo response, at least as moderators, if not mediators. The observation that improvement during non-treatment run-in predicted placebo response on two of the three endpoints, but did not predict treatment response, is consistent with the proposal that placebo-induced mechanisms, such as the release of endogenous opioids, may be activated in the anticipatory phase of the placebo response  and hence during therapist-patient or investigator-participant interaction . The variance in the predictors of placebo and active response observed in the current study is consistent with the hypothesis that different underlying mechanisms may be operating in placebo and treatment arms.
Strengths of the study include the investigation of study entry scores (2 weeks prior to commencement of run-in), in preference to baseline scores, as potential predictors of the placebo response. The effect on psychological mechanisms of enrollment in a clinical trial would be expected to occur from the point of study entry, with the initiation of investigator-participant interaction and other context effects, rather than from initiation of the intervention . However, because pharmacological effects of the intervention would only be observable from the point of administration of the intervention (see Figure 3), baseline scores were controlled for in the analysis.
A limitation in the interpretation of these findings is that there is no evidence, to our knowledge, supporting this phytotherapeutic combination as an effective treatment for menopausal symptoms. Therefore, a known pharmacological effect for this intervention in the treatment of menopausal symptoms has never been established. This study was a post hoc analysis of data from an RCT and as such, was not designed to explore neurobiological mechanisms. Thus, no definite conclusions can be drawn regarding any different mechanisms of action. Other possible limitations include the relatively small sample size, the use of exclusively subjective outcome measures, and the single scale of measurement for improvement during non-treatment run-in.
In randomised, placebo-controlled clinical trials, greater understanding of the placebo response is needed to accurately dissect out placebo versus intervention effects. In order to conclude that a pharmacological intervention is ineffective if found not to be superior to placebo, it is essential to be confident that i) the placebo has no specific effect for the condition being examined, and ii) that the effects of the placebo and active intervention are completely additive, i.e. that subtracting the placebo effect from the treatment effect leaves the active intervention effect. The assumption of additivity has previously been questioned by other authors . Early research on neuroanatomical and neurobiological mechanisms, primarily in the area of analgesia, suggests that placebo and pharmacological interventions may activate mutually exclusive pathways. The current findings could be explained in light of the theory of non-additivity. Further research is warranted into the neurophysiological basis of the placebo response to investigate the validity of the assumption of additivity. If this assumption were shown to be incorrect, it would have significant implications for the interpretation of results from placebo-controlled RCTs.
Greene Climacteric Scale
Hamilton Depression Inventory 17-item scale
Statistical Package for Social Science
MediHerb Australia Pty Ltd provided active and placebo formulations. Financial support was provided by the Australian College of Phytotherapy and the Jean Hailes Foundation for Women's Health for expenses relating to the RCT.
The authors are grateful to Eldho Paul and Dr John Reece for statistical support, Prof Marc Cohen for co-supervision of the RCT, Dr Iggy Soosay for acting as trial safety monitor, Williamina van Die for data checking, Prof Ken Greenwood for manuscript review, the late Dr. J.G. Greene for permission to reproduce Greene Climacteric Scale, and all the women who participated in the study.
Approval for the study was obtained from the RMIT University Human Research Ethics Committee.
- Kirsch I: Are drug and placebo effects in depression additive?. Biol Psychiatry. 2000, 47 (8): 733-735.View ArticlePubMedGoogle Scholar
- Kirsch I, Sapirstein G: Listening to prozac but hearing placebo: a meta-analysis of antidepressant medication. Prevention and Treatment. 1998, 1: [http://journals.apa.org/prevention/volume1/pre0010002a.html]Google Scholar
- Montgomery SA, Reimitz PE, Zivkov M: Mirtazapine versus amitriptyline in the long-term treatment of depression: a double-blind placebo-controlled study. Int Clin Psychopharmacol. 1998, 13 (2): 63-73.View ArticlePubMedGoogle Scholar
- Joffe R, Sokolov S, Streiner D: Antidepressant treatment of depression: a metaanalysis. Can J Psychiatry. 1996, 41 (10): 613-616.PubMedGoogle Scholar
- Wampold BE, Minami T, Tierney SC, Baskin TW, Bhati KS: The placebo is powerful: estimating placebo effects in medicine and psychotherapy from randomized clinical trials. J Clin Psychol. 2005, 61 (7): 835-854.View ArticlePubMedGoogle Scholar
- Leuchter AF, Cook IA, Witte EA, Morgan M, Abrams M: Changes in brain function of depressed subjects during treatment with placebo. Am J Psychiatry. 2002, 159 (1): 122-129.View ArticlePubMedGoogle Scholar
- Mayberg HS, Silva JA, Brannan SK, Tekell JL, Mahurin RK, McGinnis S, Jerabek PA: The functional neuroanatomy of the placebo effect. Am J Psychiatry. 2002, 159 (5): 728-737.View ArticlePubMedGoogle Scholar
- Leuchter AF, Morgan M, Cook IA, Dunkin J, Abrams M, Witte E: Pretreatment neurophysiological and clinical characteristics of placebo responders in treatment trials for major depression. Psychopharmacology (Berl). 2004, 177 (1–2): 15-22.View ArticleGoogle Scholar
- Colloca L, Benedetti F: Placebos and painkillers: is mind as real as matter?. Nat Rev Neurosci. 2005, 6 (7): 545-552.View ArticlePubMedGoogle Scholar
- Schultz W: Predictive reward signal of dopamine neurons. J Neurophysiol. 1998, 80 (1): 1-27.PubMedGoogle Scholar
- Benedetti F, Colloca L, Torre E, Lanotte M, Melcarne A, Pesare M, Bergamasco B, Lopiano L: Placebo-responsive Parkinson patients show decreased activity in single neurons of subthalamic nucleus. Nat Neurosci. 2004, 7 (6): 587-588.View ArticlePubMedGoogle Scholar
- Franklin KB: Analgesia and the neural substrate of reward. Neurosci Biobehav Rev. 1989, 13 (2–3): 149-154.View ArticlePubMedGoogle Scholar
- Garris PA, Kilpatrick M, Bunin MA, Michael D, Walker QD, Wightman RM: Dissociation of dopamine release in the nucleus accumbens from intracranial self-stimulation. Nature. 1999, 398 (6722): 67-69.View ArticlePubMedGoogle Scholar
- Koshi EB, Short CA: Placebo theory and its implications for research and clinical practice: a review of the recent literature. Pain Pract. 2007, 7 (1): 4-20.View ArticlePubMedGoogle Scholar
- Wager TD, Scott DJ, Zubieta JK: Placebo effects on human mu-opioid activity during pain. Proc Natl Acad Sci USA. 2007, 104 (26): 11056-11061.View ArticlePubMedPubMed CentralGoogle Scholar
- Weihrauch TR: Placebo treatment is effective differently in different diseases–but is it also harmless? A brief synopsis. Sci Eng Ethics. 2004, 10 (1): 151-155.View ArticlePubMedGoogle Scholar
- Benedetti F, Rainero I, Pollo A: New insights into placebo analgesia. Curr Opin Anaesthesiol. 2003, 16 (5): 515-519.View ArticlePubMedGoogle Scholar
- De La Fuente-Fernandez R, Stoessl AJ: The biochemical bases for reward. Implications for the placebo effect. Eval Health Prof. 2002, 25 (4): 387-398.View ArticlePubMedGoogle Scholar
- de la Fuente-Fernandez R, Schulzer M, Stoessl AJ: Placebo mechanisms and reward circuitry: clues from Parkinson's disease. Biol Psychiatry. 2004, 56 (2): 67-71.View ArticlePubMedGoogle Scholar
- de la Fuente-Fernandez R, Lidstone S, Stoessl AJ: Placebo effect and dopamine release. J Neural Transm Suppl. 2006, 415-418. 70
- Amanzio M, Benedetti F: Neuropharmacological dissection of placebo analgesia: expectation-activated opioid systems versus conditioning-activated specific subsystems. J Neurosci. 1999, 19 (1): 484-494.PubMedGoogle Scholar
- Van Die MD, Burger HG, Bone KM, Cohen MM, Teede HJ: Hypericum perforatum with Vitex agnus-castus in menopausal symptoms: a randomized, controlled trial. Menopause. 2009, 16 (1): 156-163.View ArticlePubMedGoogle Scholar
- Van Die MD, Teede H, Bone K, Reece J, Burger H: Predictors of Placebo Response in a Randomised, Controlled Trial of Phytotherapy in Menopause. Menopause. 2009Google Scholar
- Khan A, Brodhead AE, Kolts RL, Brown WA: Severity of depressive symptoms and response to antidepressants and placebo in antidepressant trials. J Psychiatr Res. 2005, 39 (2): 145-150.View ArticlePubMedGoogle Scholar
- Welge JA, Keck PE, Meinhold JM: Predictors of response to treatment of acute bipolar manic episodes with divalproex sodium or placebo in 2 randomized, controlled, parallel-group trials. J Clin Psychopharmacol. 2004, 24 (6): 607-612.View ArticlePubMedGoogle Scholar
- Heiligenstein JH, Faries DE, Rush AJ, Andersen JS, Pande AC, Roffwarg HP, Dunner D, Gillin JC, James SP, Lahmeyer H, Zajecka J, Tollefson GS, Gardner DM: Latency to rapid eye movement sleep as a predictor of treatment response to fluoxetine and placebo in nonpsychotic depressed outpatients. Psychiatry Res. 1994, 52 (3): 327-339.View ArticlePubMedGoogle Scholar
- Downing RW, Rickels K: Predictors of response to amitriptyline and placebo in three outpatient treatment settings. J Nerv Ment Dis. 1973, 156 (2): 109-129.View ArticlePubMedGoogle Scholar
- Khan A, Leventhal RM, Khan SR, Brown WA: Severity of depression and response to antidepressants and placebo: an analysis of the Food and Drug Administration database. J Clin Psychopharmacol. 2002, 22 (1): 40-45.View ArticlePubMedGoogle Scholar
- Enck P, Vinson B, Malfertheiner P, Zipfel S, Klosterhalfen S: The placebo response in functional dyspepsia – reanalysis of trial data. Neurogastroenterol Motil. 2009, 21 (4): 370-377.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/9/41/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.