Mediation analysis of the relationship between institutional research activity and patient survival
© Rochon et al.; licensee BioMed Central Ltd. 2014
Received: 1 August 2013
Accepted: 14 January 2014
Published: 22 January 2014
Recent studies have suggested that patients treated in research-active institutions have better outcomes than patients treated in research-inactive institutions. However, little attention has been paid to explaining such effects, probably because techniques for mediation analysis existing so far have not been applicable to survival data.
We investigated the underlying mechanisms using a recently developed method for mediation analysis of survival data. Our analysis of the effect of research activity on patient survival was based on 352 patients who had been diagnosed with advanced ovarian cancer at 149 hospitals in 2001. All hospitals took part in a quality assurance program of the German Cancer Society. Patient outcomes were compared between hospitals participating in clinical trials and non-trial hospitals. Surgical outcome and chemotherapy selection were explored as potential mediators of the effect of hospital research activity on patient survival.
The 219 patients treated in hospitals participating in clinical trials had more complete surgical debulking, were more likely to receive the recommended platinum-taxane combination, and had better survival than the 133 patients treated in non-trial hospitals. Taking into account baseline confounders, the overall adjusted hazard ratio of death was 0.58 (95% confidence interval: 0.42 to 0.79). This effect was decomposed into a direct effect of research activity of 0.67 and two indirect effects of 0.93 each mediated through either optimal surgery or chemotherapy. Taken together, about 26% of the beneficial effect of research activity was mediated through the proposed pathways.
Mediation analysis allows proceeding from the question “Does it work?” to the question “How does it work?” In particular, we have shown that the research activity of a hospital contributes to superior patient survival through better use of surgery and chemotherapy. This methodology may be applied to analyze direct and indirect natural effects for almost any combination of variable types.
KeywordsTrial effect Research activity Healthcare outcomes Mediation Survival analysis
Well-conducted trials are critical to advancements in oncology. However, such trials require patients and healthcare providers who are willing to participate in clinical research. The effect of participation in clinical trials is controversially discussed in the literature. Reviews on this effect have been published in the past, but these have usually focused on comparisons between ‘trial patients’ and ‘non-trial patients’ and authors have come to conflicting conclusions [1–3].
In the last years another relevant aspect of participation in clinical trials has emerged: “Do healthcare institutions or service providers who are active in research deliver better care and outcomes than those who do not participate in clinical research?” , p. 6]. According to Pater et al. , especially from the healthcare policy perspective, this question should be the main focus of future research because benefit from a research-active institution will affect many more patients than just the small proportion of participants in clinical trials. In addition, this issue may also be of greater importance to patients who have to decide where to go to receive healthcare and by whom. However, only one review conducted by Clarke and Loudon  included a literature search to identify studies addressing the relationship between institutional research activity and patient outcomes. The available evidence suggests that patients benefit from being treated by practitioners or in institutions participating in trials. The authors concluded that research activity might result in better health care outcomes, although the reason for this effect remains unclear. Indeed, most studies focus on answering the pragmatic question of whether research activity has an effect on patient outcome. However, little attention is being paid to the explanation of such effects (“How does it work?”, i.e. the assessment of mediation).
For normally distributed mediators and outcomes and in the absence of interactions or non-linear effects, natural direct and indirect effects can be estimated by a stepwise approach based on standard linear regressions . In short, models are fitted for the outcome both with and without the mediator, and the difference in the coefficients for the exposure is taken as a measure of the indirect or mediated effect. In oncology, however, the outcome of interest is often patient survival. Survival times are known to be non-normally distributed and typically right censored. Therefore, existing techniques for mediation analysis are not applicable to survival data, and several authors have shown that the linear regression approach cannot be simply translated to logistic regression or proportional hazard models [10–12].
The aim of this article is to introduce a recently developed methodology that can be used to assess mediation on all outcome types including survival data, and to use this method to explain the effects of institutional research activity on ovarian cancer patient survival.
The study is part of an ongoing quality assurance program (QS-OVAR) initiated in 1999 by the Arbeitsgemeinschaft Gynaekologische Onkologie (AGO) Organkommission OVAR, a subcommittee of the German Cancer Society. The aim of this program is to describe the pattern and quality of care of patients with ovarian cancer in Germany as well as to improve their outcome. The hypothesis is that research-active hospitals yield improved outcomes for their patients. Research activity at hospital level is defined as hospital participation in prospective clinical trials conducted by one of the two German cooperative study groups, the AGO Study Group Ovarian Cancer (AGO-OVAR) and the Northeastern Society of Gynecologic Oncology (NOGGO). More specifically, all patients treated in hospitals participating in clinical trials are compared with all patients treated in hospitals that did not participate in any clinical trials of the AGO-OVAR and NOGGO.
The mediation analysis of the effect of research activity on healthcare outcomes is illustrated through QS-OVAR 2001. This study involved 165 hospitals and 476 patients with early and advanced ovarian cancer diagnosed in the third quarter of 2001. The primary outcome measure was overall survival. Secondary outcomes were the receipt of standard care with regard to surgery and chemotherapy. Details on study design and results have been described elsewhere [13, 14]. Here, we will focus on patients with advanced ovarian cancer because most death events in QS-OVAR 2001 occurred in patients with advanced stages Fédération Internationale de Gynécologie et d'Obstétrique (FIGO) IIB–IV, whereas only a few events were observed in FIGO I–IIA patients. Furthermore, treatment options differ markedly between early and advanced stage disease. According to the German guidelines, patients with ovarian cancer FIGO IIB and higher should receive surgery including maximal debulking as well as chemotherapy with carboplatin and paclitaxel . We thus considered two potential mediators. First, ‘optimal debulking’ with postoperative tumor residuals up to 1 cm was regarded as standard care with respect to surgery (i.e. ‘optimal surgery’). Second, each platinum-taxane combination was considered adherent to treatment guidelines regarding chemotherapy (i.e. ‘optimal chemotherapy’).
The use of data included in this study was approved by the AGO-OVAR and the AGO Ovarian Committee. All data was anonymized prior to analysis. The study was approved by the Ethics Committee of the Medical Faculty of the University of Heidelberg, Germany (study number: S-446/2013).
All analyses were conducted in R version 3.0.2 . Continuous data were summarized with median and interquartile range (IQR), categorical data by counts and percentages. Survival curves were generated by the Kaplan-Meier method. The Cox proportional hazards model with robust variance estimator was used to assess the relationship between hospital participation in clinical trials and overall survival. Logistic regression models were fitted for ‘optimal surgery’ and ‘optimal chemotherapy’ with generalized estimating equations to account for clustering of patients within hospitals. The following patient and disease characteristics were assumed to control for confounding: age at diagnosis (continuous, in 5 years units), Eastern Cooperative Oncology Group (ECOG) performance status (>1 vs. 0/1), ascites (> 500 ml vs. ≤ 500 ml), comorbidity (present vs. none), histology (serous vs. other), and grade (G 3/4 vs. G 1/2).
Mediation analysis was performed using the approach proposed by Lange et al. . This approach is based on the counterfactual framework  and allows decomposition of the total effect of a given exposure A on the outcome Y into a natural direct effect (A → Y in Figure 1A) and a natural indirect effect through a mediator M (A → M → Y in Figure 1A). In case of a time-to-event outcome Y, a binary exposure A, a binary mediator M and a number of baseline confounders C, Lange et al. showed that unbiased estimates for the direct and indirect effect are obtained from weighted Cox regression of the time-to-event outcome on A, A* and C using a duplicated data set. In the first replication A* takes the original value of the exposure. In the second replication A* takes the opposite (‘counterfactual’) value of the exposure. The weights are determined by W c = P(M | A*, C) / P(M | A, C), with P( · ) deriving from a logistic regression of the mediator M on the exposure and the baseline confounders [17, Appx. 4]. Assuming non-informative censoring and proportional hazards, the weighted Cox model then yields hazard ratios for A and A* that serve as estimates for the natural direct effect and indirect effect, respectively. The product of the two hazard ratios yields the hazard ratio for the total effect. Standard errors and confidence intervals can, for example, be determined by bootstrap methods.
In our study, we explored two binary mediators (surgery and chemotherapy) of the effect of hospital research activity on survival (Figure 1B). Under the assumption of separate causal pathways through the two mediators, unbiased point estimates for the natural direct effect and the natural indirect effects related to the two mediators were obtained by a weighted Cox regression of the outcome on the exposure, the baseline confounders and two additional counterfactual variables A1* and A2* that were systematically manipulated in four replicates of the original data . Confidence intervals for mediation effects that account for clustering of patients within hospitals were obtained using simple random cluster sampling and 10,000 bootstrap simulations. In all analyses, results were considered statistically significant if the 95% confidence interval (CI) for the hazard ratio (HR) or odds ratio (OR) did not include 1.
In the additional material (see Additional file 1), we have provided a detailed description of the analysis with the corresponding R code to enable future researchers to assess mediation in a survival context. There, we also show how we tested for interactions between exposure and confounders, how we assessed the linearity assumption for all variables, and how we allowed for misspecification in the mediators. Finally, this description includes sensitivity analyses with only one mediator representing adherence to treatment guidelines with regard to surgery and chemotherapy as a binary and an ordinal variable.
Patient and disease characteristics at diagnosis
No. of patients, N (%)
Performance status, N (%)
ECOG > 1
Ascites, N (%)
≤ 500 ml
> 500 ml
Comorbidity, N (%)
Histology, N (%)
Grade, N (%)
Multivariable Cox regression analysis for overall survival in patients with advanced ovarian cancer
Continuous (5 years)
ECOG > 1
≤ 500 ml
> 500 ml
In the next step, we explored the effect of standard care on survival, again by means of a multivariable Cox model. As expected, both potential mediators were associated with survival. The adjusted HR of death for optimally debulked patients was 0.46 (95% CI: 0.34 to 0.63) compared to patients with a postoperative tumor residual larger than 1 cm. Similarly, we found a survival benefit for patients treated with platinum-taxane chemotherapy (adjusted HR = 0.42, 95% CI: 0.28 to 0.62).
Finally, we used the approach of Lange et al.  to test if the observed positive effect of institutional research activity on patient survival is at least partially mediated through surgery and chemotherapy (Figure 1B).
Mediation analysis: Total, direct and indirect effects with 95% confidence intervals
Effect (of trial hospital vs. non-trial hospital)
Indirect effect (through surgery)
Indirect effect (through chemotherapy)
Indirect effect (through both surgery and chemotherapy)
The effect of institutional research activity on patient outcomes has not yet been investigated extensively, despite its great relevance to healthcare providers, policy makers, and patients. So far, only a few studies have examined the association between patient outcomes and institutional participation in clinical trials, as opposed to trial participation of individual patients . Thus, the authors of the 2011 special issue of Annals of Oncology entitled “Clinical Research and Healthcare Outcomes: A Workshop at the International Agency for Research on Cancer, Lyon” agreed that further research on this topic is urgently required. If research activity has beneficial effects on patient outcomes, these benefits are not solely restricted to research participants.
Investigating the relationship between research activity at the level of institutions or providers and outcomes at the level of patients is not as easy as it seems at first glance. Researchers interested in this relationship are confronted with several practical challenges, such as getting data from less research-active institutions, as well as with several methodological challenges, for instance, the choice of the appropriate study design . In addition, sole focus on establishing effects and measuring the potential benefits of research activity on healthcare outcomes is often considered insufficient . The assessment of variables mediating the effects may be helpful in explaining the effects of exposure or in investigating the reasons why an exposure failed to yield an expected outcome. More importantly, knowledge of such mechanisms enables specific measures to improve healthcare at the individual level and at the institutional level—and such measures may be implemented even in the absence of exposure. Krzyzanowska et al.  described a conceptual framework for understanding how institutional research activity might lead to better outcomes, even for patients who are not participants in a research project. The authors pointed out that the processes of care received by patients may have a strong impact on outcomes and that such processes may systematically differ between research-active and research-inactive settings. For example, institutions that actively participate in research may be more likely to follow clinical guidelines for cancer treatment. In addition, participation in research may facilitate early access to new treatment approaches, allowing the faster implementation of new evidence into practice.
The main purpose of the present work was to explore the mechanisms underlying the association between institutional research activity and patient outcomes. We first re-examined the association between institutional research activity and survival described by du Bois et al. . Using the data of 352 patients with advanced ovarian cancer, we showed that hospital participation in clinical trials was associated with improved survival. The effect observed in our study was large and clearly relevant because of a relative reduction in the risk of death by 42% in favor of research activity. Whether this effect is in line with the literature is difficult to answer because, with respect to ovarian cancer, trial participation has hardly been investigated outside Germany. Furthermore, only a few studies have examined the association between patient outcomes and research activity at the level of health care institutions .
As expected, patients with ‘optimal surgery’ and ‘optimal chemotherapy’ lived longer than patients who were not treated according to the standard of care. In addition, patients treated in trial hospitals were more likely to receive optimal treatment than patients treated in non-trial hospitals. We thus confronted the question of how much of the effect of hospital participation in clinical trials on patient survival was mediated through optimal debulking and optimal chemotherapy selection. To answer this question, we used a recently developed methodology for assessing mediation in the context of a survival analysis . Taking into account several known baseline confounders, the overall hazard ratio (total effect) of 0.58 was decomposed into a direct effect of research activity of 0.67 and two indirect effects of 0.93 each mediated through surgery and chemotherapy. The aggregate indirect effect through both mediators was 0.87, that is, about 26% of the beneficial effect of research activity was mediated through both surgery and chemotherapy. In conclusion, trial participation of a hospital contributed at least partially to a superior outcome through the better quality of treatment provided.
The probability of surviving ovarian cancer depends on (1) patient characteristics (e.g. age), (2) tumor biology (e.g. stage), and (3) the quality of treatment (e.g. surgical outcome, selection of chemotherapy regimen). The first two factors are hard to change but the quality of treatment is susceptible to direct influence and thus seems to be of utmost relevance when considering efforts to improve the outcome of this disease. The implementation of standards into clinical routine in our study was not satisfactory and still needs improvement; however, taken both mediators together, treatment standards were more strictly implemented in trial hospitals than in non-trial hospitals. The good news is that clinicians can influence the implementation of standards, regardless of whether they are employed in research-active hospitals or not.
The main limitation of our study is the lack of randomization of patients into research-active and research-inactive hospitals. Such a randomized controlled trial would facilitate a clear causal interpretation but would also be hard to implement. When randomization is not possible, well-conducted observational studies can provide the necessary data to guide the future development of clinical research and healthcare. In particular, mediation analysis can help explore the mechanisms by which research activity leads to the outcome of interest. The key assumption for mediation analysis is that all relevant confounders are included into the analysis. To be more precise, the approach proposed by Lange et al.  requires that the considered variables are sufficient for controlling the confounding of 1) the exposure-outcome relation, 2) the exposure-mediator relation, and 3) the mediator-outcome relation. We addressed this issue by incorporating all established prognostic factors into the models for the mediator and the outcome. In particular, age at diagnosis, FIGO stage, ECOG performance status, volume of ascites, histology, grade, and comorbidity are well-known prognostic factors for survival in ovarian cancer [22–24]. These factors also influence treatment recommendations.
There must be no other variables (measured or unmeasured) that confound the mediator-outcome relation which are themselves affected by the exposure; this last assumption essentially ensures that each of the considered pathways between exposure and outcome does indeed represent a unique causal mechanism. In observational studies, these assumptions are inherently untestable and must instead be justified by means of knowledge about the biological processes under consideration. In our study, for example, we did not collect information on socioeconomic factors (e.g. income or insurance status). Patients with a higher socioeconomic status (SES) may find it easier and hence choose to travel to specialized or better rated hospitals or institutions that are research-active and participate in clinical trials. In contrast, cancer patients with a low SES may be less likely to choose research-active hospitals because of the lack of corresponding health care information. The total effect of research activity would then at least be partially due to better general survival rates of patients with a high SES. Lower socioeconomic status has occasionally been associated with lower likelihood of receiving surgery and chemotherapy but does not seem to be an independent prognostic factor for survival in ovarian cancer [25, 26].
Finally, the mediation analysis is only valid if the logistic regressions are adequate descriptions of the mediators. Sensitivity analyses showed that misclassification of the mediator will bias the estimates of the indirect effect towards one and the direct effect away from one. In our study, measurement error was minimized by objectively assessing the quality of chemotherapy and by evaluating surgical outcome (i.e. tumor residual) in a standardized manner.
Mediation analysis, which is regularly and successfully applied elsewhere in health care, has a great potential to be used as an instrument for investigating underlying mechanisms. This tool allows analysis to go beyond the question “Does it work?” In particular, mediation analysis has shown that the research activity of a hospital contributes to superior patient survival through better adherence to treatment guidelines with regard to surgical outcome and selection of chemotherapy. Yet, the analysis of the ovarian cancer study presented in this article demonstrates only one way of applying this methodology, but the potential for this technique is much wider. This methodology, which can be conducted with standard software, may be applied to analyze direct and indirect effects for almost any combination of variable types (in particular, the mediator and outcome can be binary, ordinal, categorical, or continuous, and the outcome can also be survival time). Researchers interested in the question “How does it work?” are thus strongly encouraged to use mediation analysis in their investigations.
The AGO quality assurance program QS-OVAR 2001 was supported with a grant by Bristol–Myers–Squibb, Germany.
- Braunholtz DA, Edwards SJ, Lilford RJ: Are randomized clinical trials good for us (in the short term)? Evidence for a ‘trial effect’. J Clin Epidemiol. 2001, 54: 217-224. 10.1016/S0895-4356(00)00305-X.View ArticlePubMed
- Peppercorn JM, Weeks JC, Cook EF, Joffe S: Comparison of outcomes in cancer patients treated within and outside clinical trials: conceptual framework and structured review. Lancet. 2004, 363: 263-270. 10.1016/S0140-6736(03)15383-4.View ArticlePubMed
- Vist GE, Bryant D, Somerville L, et al: Outcomes of patients who participate in randomized controlled trials compared to similar patients receiving similar interventions who do not participate. Cochrane Database Syst Rev. 2008, 3: MR000009
- Selby P, Autier P: The impact of the process of clinical research on health service outcomes. Ann Oncol. 2011, 22 (Suppl 7): vii5-vii9.View ArticlePubMed
- Pater J, Rochon J, Parmar M, Selby P: Future research and methodological approaches. Ann Oncol. 2011, 22 (Suppl 7): vii57-vii61.View ArticlePubMed
- Clarke M, Loudon K: Effects on patients of their healthcare practitioner’s or institution’s participation in clinical trials: a systematic review. Trials. 2011, 12: 16-10.1186/1745-6215-12-16.PubMed CentralView ArticlePubMed
- Baron RM, Kenny DA: The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. J Pers Soc Psychol. 1986, 51: 1173-1182.View ArticlePubMed
- Robins JM, Greenland S: Identifiability and exchangeability for direct and indirect effects. Epidemiology. 1992, 3: 143-155. 10.1097/00001648-199203000-00013.View ArticlePubMed
- Hafeman DM, Schwartz S: Opening the Black Box: a motivation for the assessment of mediation. Int J Epidemiol. 2009, 38: 838-845. 10.1093/ije/dyn372.View ArticlePubMed
- Cole SR, Hernán MA: Fallibility in estimating direct effects. Int J Epidemiol. 2002, 31: 163-165. 10.1093/ije/31.1.163.View ArticlePubMed
- Kaufman JS, Maclehose RF, Kaufman S: A further critique of the analytic strategy of adjusting for covariates to identify biologic mediation. Epidemiol Perspect Innov. 2004, 1: 4-10.1186/1742-5573-1-4.PubMed CentralView ArticlePubMed
- VanderWeele TJ: Causal mediation analysis with survival data. Epidemiology. 2011, 22: 582-585. 10.1097/EDE.0b013e31821db37e.PubMed CentralView ArticlePubMed
- du Bois A, Rochon J, Lamparter C, Pfisterer J: Pattern of care and impact of participation in clinical studies on the outcome in ovarian cancer. Int J Gynecol Cancer. 2005, 15: 183-191. 10.1111/j.1525-1438.2005.15202.x.View ArticlePubMed
- Rochon J, du Bois A: Clinical research in epithelial ovarian cancer and patients’ outcome. Ann Oncol. 2011, 22 (Suppl 7): vii16-vii19.View ArticlePubMed
- Bauknecht T, Breitbach GP, du Bois A, et al: Maligne Ovarialtumoren. Diagnose und Therapie maligner Erkrankungen. Kurzgefasste interdisziplinaere Leitlinien. Edited by: Hermanek P. 2000, Muenchen: Zuckschwerdt Verlag, 301-318.
- R Core Team: R: A language and environment for statistical computing. 2013, Vienna, Austria: R Foundation for Statistical Computing
- Lange T, Vansteelandt S, Bekaert M: A simple unified approach for estimating natural direct and indirect effects. Am J Epidemiol. 2012, 176: 190-195. 10.1093/aje/kwr525.View ArticlePubMed
- Pearl J: Causality: Models, Reasoning, and Inference. 2009, New York, NY: Cambridge University PressView Article
- Lange T, Rasmussen M, Thygesen LC: Assessing natural direct and indirect effects through multiple pathways. Am J Epidemiol. 2014, 179: 513-518. 10.1093/aje/kwt270.View ArticlePubMed
- Pfisterer J, Weber B, Reuss A, et al: Randomized phase III trial of topotecan following carboplatin and paclitaxel in first-line treatment of advanced ovarian cancer: A gynecologic cancer intergroup trial of the AGO-OVAR and GINECO. J Natl Cancer Inst. 2006, 98: 1036-1045. 10.1093/jnci/djj296.View ArticlePubMed
- Krzyzanowska MK, Kaplan R, Sullivan R: How may clinical research improve healthcare outcomes?. Ann Oncol. 2011, 22 (Suppl 7): vii10-vii15.View ArticlePubMed
- Thigpen T, Brady MF, Omura GA, et al: Age as a prognostic factor in ovarian carcinoma. The Gynecologic Oncology Group experience. Cancer. 1993, 71 (2 Suppl): 606-614.PubMed
- Clark TG, Stewart ME, Altman DG, et al: A prognostic model for ovarian cancer. Br J Cancer. 2001, 85: 944-952. 10.1054/bjoc.2001.2030.PubMed CentralView ArticlePubMed
- Sperling C, Noer MC, Christensen IJ, et al: Comorbidity is an independent prognostic factor for the survival of ovarian cancer: a Danish register-based cohort study from a clinical database. Gynecol Oncol. 2013, 129: 97-102. 10.1016/j.ygyno.2012.12.039.View ArticlePubMed
- Sugiyama VE, Shin JY, Kapp DS, et al: The effect of socioeconomic status on the survival of ovarian cancer patients. J Clin Oncol. 2008, 26 (Suppl): 5557-
- Bristow RE, Powell MA, Al-Hammadi N, et al: Disparities in ovarian cancer care quality and survival according to race and socioeconomic status. J Natl Canc Inst. 2013, 105: 823-832. 10.1093/jnci/djt065.View Article
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/14/9/prepub
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.