This article has Open Peer Review reports available.
Protocol adherence for continuously titrated interventions in randomized trials: an overview of the current methodology and case study
- F. Lauzier1, 2, 3,
- N. K. Adhikari4, 5,
- A. Seely6, 7,
- K. K. Y. Koo8, 9,
- E. P. Belley-Côté10, 11,
- K. E. A. Burns4, 12,
- D. J. Cook10,
- F. D’Aragon13, 14,
- B. Rochwerg10,
- M. E. Kho15,
- S. J. W. Oczkowksi10,
- E. H. Duan10,
- M. O. Meade10,
- A. G. Day16,
- F. Lamontagne11, 14Email author and
- for ACCADEMY (Academy of Critical Care – Development, Evaluation, and Methodology)
© The Author(s). 2017
Received: 22 February 2017
Accepted: 10 July 2017
Published: 17 July 2017
The standard definition for protocol adherence is the proportion of all scheduled doses that are delivered. In clinical research, this definition has several limitations when evaluating protocol adherence in trials that study interventions requiring continuous titration.
Building upon a specific case study, we analyzed a recent trial of a continuously titrated intervention to assess the impact of different definitions of protocol deviations on the interpretation of protocol adherence. The OVATION pilot trial was an open-label randomized controlled trial of higher (75–80 mmHg) versus lower (60–65 mmHg) mean arterial pressure (MAP) targets for vasopressor therapy in shock. In this trial, potential protocol deviations were defined as MAP values outside the targeted range for >4 consecutive hours during vasopressor therapy without synchronous and consistent adjustments of vasopressor doses. An adjudication committee reviewed each potential deviation to determine if it was clinically-justified or not. There are four reasons for this contextual measurement and reporting of protocol adherence. First, between-arm separation is a robust measure of adherence to complex protocols. Second, adherence assessed by protocol deviations varies in function of the definition of deviations and the frequency of measurements. Third, distinguishing clinically-justified vs. not clinically-justified protocol deviations acknowledges clinically sensible bedside decision-making and offers a clear terminology before the trial begins. Finally, multiple metrics exist to report protocol deviations, which provides different information but complementary information on protocol adherence.
In trials of interventions requiring continuous titration, metrics used for defining protocol deviations have a considerable impact on the interpretation of protocol adherence. Definitions for protocol deviations should be prespecified and correlated with between-arm separation, if it can be measured.
In clinical practice, adherence is defined as the extent to which a person’s behaviour corresponds with the recommendations made by a healthcare provider [1, 2]. In a research setting, non-adherence hinders the feasibility of clinical trials because patients who do not receive the intended intervention will not be affected by it . In addition, non-adherence may bias trial results because patients who do not adhere to study protocols may have an inherently different prognosis compared to those who comply with the protocol . Accordingly, variations in how protocol adherence is monitored and reported may bear on the interpretation of clinical trials [5–7].
The standard definition for protocol adherence is the proportion of all scheduled doses that are received . This approach, however, is not applicable to interventions requiring continuous titration where a total number of scheduled doses is irrelevant. With interventions like continuous infusions of vasopressors and insulin, as well as mechanical ventilation, there exist no finite number of scheduled doses. In this article, we propose an approach to measure and report adherence in trials of continuous interventions titrated by bedside teams. This approach, informed by a recent pilot trial, acknowledges clinically-justified deviations and contextualizes protocol adherence into a pragmatic framework.
Case study – The OVATION pilot trial
The OVATION pilot trial was an open-label randomized controlled trial of higher versus lower mean arterial pressure (MAP) targets for vasopressor therapy in shock . The trial interventions were developed with input from clinicians and, with Research Ethics Board approval at each of 11 participating academic hospitals from Canada and the United States, we randomly assigned critically ill patients with vasodilatory shock to a lower (60–65 mmHg) versus higher (75–80 mmHg) MAP target. The primary feasibility objective of the trial was to achieve a 5 mmHg difference in MAP during vasopressor therapy between study groups. Intensive care unit (ICU) research personnel screened and enrolled eligible patients and recorded MAP and vasopressor doses hourly for the duration of vasopressor therapy. This approach enabled the investigators to measure the time within, above, and below assigned target MAP ranges, and to identify whether adjustments to vasopressor infusions complied with the study protocol. From May 2013 to April 2014, 118 patients participated in the OVATION pilot trial and 117 received vasopressor infusions for 695 patient-days.
In this trial, we defined potential protocol deviations a priori as MAP values outside the targeted range for >4 consecutive hours during vasopressor therapy without synchronous and consistent adjustments of vasopressor doses. As such, out of range MAPs did not constitute protocol deviations when the bedside team titrated vasopressors as required (i.e., increased dose if MAP was below the prescribed range or decreased dose if MAP was too high during the 4-h window). This 4-h time window was proposed by the steering committee and approved by all OVATION trial investigators. Moreover, we adjudicated each potential protocol deviation to determine if these events were clinically justified (Additional file 1). This approach was in sharp contrast to the mere reporting of MAP values at selected time points. Herein, we discuss four advantages of this contextual measurement and reporting of protocol adherence in clinical trials of interventions that require continuous titration.
Reason 1: Between-arm separation is a robust measure of adherence to complex protocols.
While protocol deviations may occur with simple interventions, they are more likely to occur in complex interventions. Thus, interventions requiring frequent or continuous titration present the greatest challenge, since protocol deviations may occur at any moment of the trial, in contrast to once or at most several times per day for most other interventions. When designing the OVATION pilot trial, it was not possible to specify an acceptable number of ‘missed doses’ for continuous vasopressor infusions. Instead, the extent of separation in MAP between arms was chosen as a crucial feasibility criterion to proceed to a larger trial. Accordingly, we prespecified what we considered the minimally acceptable separation based on clinical considerations, recognizing that even smaller differences (e.g. difference in MAP <5 mmHg) may be statistically significant but clinically unimportant. We observed a mean difference of 9 mmHg (95% confidence interval: 7 to 11 mmHg; p < 0.01) in average MAP between arms on days with vasopressors administered, thus exceeding the predefined threshold for feasibility of 5 mmHg.
Reason 2: Adherence assessed by protocol deviations varies in function of the definition of deviations and the frequency of measurements.
The ‘80% rule’ refers to the proportion of all scheduled doses that are received and is often cited as the cutoff for acceptable adherence for pharmacological interventions . As mentioned above, with continuous interventions the number of scheduled doses is not finite. As the frequency of monitoring increases, so do the resources required to track protocol adherence. Close monitoring of protocol deviations is unlikely in trials that lack resources, introducing a risk of ascertainment bias. In contrast, with very frequent monitoring of continuous interventions, the likelihood of recording protocol deviations increases but the clinical impact of each deviation diminishes proportionately. Similarly, criteria for protocol deviations that are too sensitive would exaggerate the impact of protocol deviations beyond what is clinically important and unduly undermine the apparent feasibility and internal validity of the trial.
Protocol adherence according to different definitions
N = 58
N = 59a
a. Proportion of total hours on vasopressors in and out of rangeb
Each patient weighted equally
Time within range
Patients weighted proportionally to total hours on VP
Time within range
b. MAP out of range for four consecutive vasopressor hours
Number of occurrences of four consecutive hours/vasopressor days (mean events per day)
No important differences
Number of days with MAP out of range for at least four consecutive hours (%)
Number of patients with at least one occurrence of MAP out of range for four consecutive hours (%)
c. MAP values outside the targeted range for four consecutive hours without synchronous and consistent adjustments of vasopressor doses
Number of deviation events (mean events per day)
Number of days with at least one deviation event (%)
Number of patients with at least one deviation event (%)
Reason 3: Distinguishing clinically-justified vs. not clinically-justified protocol deviations acknowledges clinically sensible bedside decision-making and offers a clear terminology before the trial begins.
Reason 4: Multiple metrics exist to report protocol deviations.
The traditional approach captures the proportion of patients who receive an arbitrary number of scheduled doses. A thoughtful approach is needed to understand the range of reporting options available and to select a metric for reporting protocol deviations that align with the trial objectives. In addition to the criteria that define protocol deviations, the units of measurement also influence the measure of adherence. Choosing to report the number of protocol deviation events, the number of days with at least one deviation event or the number of patients who ever experienced one protocol deviation provides different information. In the OVATION pilot trial, adherence was similar in both arms using the last definition, but was superior in the high MAP arm when we used the first two definitions (Table 1).
Building on previous work
Various terminologies describe the numerous issues related to protocol adherence. Ellis et al. have categorized non-adherence as ‘erratic adherence’, ‘unwitting non-adherence’ and ‘intelligent non-adherence’ . In some trials, the cessation of a trial intervention prompted by such an event could be considered as a clinically sensible decision with no further comment, a protocol deviation, a protocol violation, or even a serious adverse event without explicit reference to protocol adherence. The distinction between clinically-justified deviations that are not avoidable from the trial perspective and those that are not clinically-justified can be useful to researchers and informed readers of the medical literature . Central and, ideally, blinded adjudication of clinical events and associated protocol decisions can help make these distinctions.
Strengths and limitations
The strengths of this analysis include a priori definitions for protocol deviations, pilot trial feasibility criteria based on between-arm separation in MAP and total vasopressor dose, as well as parallel comparisons between various metrics. One limitation is that our observations are nested in a single, small clinical trial comparing complex critical care interventions and, thus, do not apply to all settings. Our intention was not to address every methodological issue that relates to protocol adherence. Rather, we used the OVATION pilot trial as a case study to illustrate that failure to contextualize the monitoring, reporting and interpretation of protocol adherence may lead to inappropriate conclusions. Another potential limitation is that there is no objective gold standard to determine the suitability of alternative approaches to the monitoring and reporting of protocol adherence. Admittedly, the definitions of protocol deviations in the OVATION pilot trial were arbitrary (e.g. 4-h time window). However, this is in keeping with every other measure of acceptable adherence, such as the 80% rule, which are typically well accepted. Our intention was not to present a superior definition for protocol adherence. Rather, the purpose of this analysis was to exemplify how trialists who are mindful of the context in which a complex research protocols are delivered may plan the monitoring of protocol adherence. While a clearer taxonomy of issues related to protocol adherence would undoubtedly be valuable to trialists and readers of the medical literature, this project was not designed to achieve this goal. Finally, our work focused on current issues related to protocol adherence. In the future, with the rapid development of real-time, data driven algorithms, the nomenclature for protocol adherence may yet become even more complex.
Metrics used for defining protocol deviations in a trial evaluating an intervention requiring continuous titration can significantly modify the interpretation of protocol adherence. Protocol deviations should be defined a priori and adapted to the complexity of the clinical context. With more complex interventions, an adjudication process to determine whether the deviations encountered were clinically justified may be warranted.
F Lauzier and F Lamontagne are recipients of research career awards from the Fonds de la recherche du Québec-Santé. DJ Cook and ME Kho each hold a Canada Research Chair. We would like to thank (in alphabetical order) the research coordinators involved in this trial: Sarah-Judith Breton, Katia Cauchon, Paulina Farias, Susan Fleury, Brittany Giacomino, Marnie Jakab, Lisa Julien, Catherine Krause, Chantal Langevin, Heather Langlois, Nicole Marinoff, Andrea Matte, Tracy McArdle, Katie O’Brien, Rebecca Porteus, Sumesh Shah, Samantha Taylor, and Irene Watpool. We thank Marie-Claude Battista, Marie-Ève Thibault and the Unité de Recherche Clinique et Épidémiologique (URCE) of the Centre de recherche du CHU de Sherbrooke for their support in coordinating the preparation and revisions of this paper.
There has been no significant financial support for this work that could have influenced its outcome.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Study concept: FL. Study design: FL, NKA, EPB-C, KB, DJC, FD’A, BR, MEK, SJWO, EHD, MOM, FL. Acquisition of data: FL, AS, KKYK, FL. Analysis of data: AD, FL. Interpretation of data: FL, NKA, EPB-C, KB, DJC, FD’A, BR, MEK, SJWO, EHD, MOM, FL. Drafting of the manuscript: FL, FL. Revising of the manuscript for critically important intellectual content: NKA, AS, KKYK, EPB-C, KB, DJC, FD’A, BR, MEK, SJWO, EHD, MOM, AD, FL. All authors have read and approved the final manuscript.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Haynes R. Introduction. Compliance in Health Care. Baltimore: Johns Hopkins University Press; 1979. p. 1–7.Google Scholar
- Dodd S, White IR, Williamson P. Nonadherence to treatment protocol in published randomised controlled trials: a review. Trials. 2012;13:84. doi:10.1186/1745-6215-13-84.View ArticlePubMedPubMed CentralGoogle Scholar
- Goldsmith C. The effect of compliance distributions on therapeutic trials. Baltimore. In: Press JHU, editor. Compliance in Health Care. 1979: 297–308.Google Scholar
- Haynes RB, Dantes R. Patient compliance and the conduct and interpretation of therapeutic trials. Control Clin Trials. 1987;8(1):12–9.View ArticlePubMedGoogle Scholar
- Haynes R, Sackett D, Guyatt G, Tugwell P. Clinical Epidemiology: How to Do Clinical Practice Research. New York. Lippincott Williams & Wilkins; 2006.Google Scholar
- Chan AW, Tetzlaff JM, Gotzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586. doi:10.1136/bmj.e7586.View ArticlePubMedPubMed CentralGoogle Scholar
- Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869. doi:10.1136/bmj.c869.View ArticlePubMedPubMed CentralGoogle Scholar
- Cramer JA, Roy A, Burrell A, Fairchild CJ, Fuldeore MJ, Ollendorf DA, et al. Medication compliance and persistence: terminology and definitions. Value Health. 2008;11(1):44–7. doi:10.1111/j.1524-4733.2007.00213.x.View ArticlePubMedGoogle Scholar
- Lamontagne F, Meade MO, Hebert PC, Asfar P, Lauzier F, Seely AJ, et al. Higher versus lower blood pressure targets for vasopressor therapy in shock: a multicentre pilot randomized controlled trial. Intensive Care Med. 2016;42(4):542–50. doi:10.1007/s00134-016-4237-3.View ArticlePubMedGoogle Scholar
- Ellis S, Shumaker S, Sieber W, Rand C. Adherence to pharmacological interventions. Current trends and future directions. The pharmacological intervention working group. Control Clin Trials. 2000;21(5 Suppl):218S–25S.View ArticlePubMedGoogle Scholar
- Zhang Z, Peluso MJ, Gross CP, Viscoli CM, Kernan WN. Adherence reporting in randomized controlled trials. Clin Trials. 2014;11(2):195–204. doi:10.1177/1740774513512565.View ArticlePubMedGoogle Scholar
- Kavanagh BP, Nurok M. Standardized Intensive Care. Protocol misalignment and impact misattribution. Am J Respir Crit Care Med 2016;193(1):17-22. doi:10.1164/rccm.201502-0314CP.
- Lamontagne F, Cook DJ, Adhikari NKJ, Briel M, Duffett M, Kho ME et al. Vasopressor administration and sepsis: A survey of Canadian intensivists. Journal of Critical Care. 2011;26(5). doi:10.1016/j.jcrc.2011.01.005.
- Bhatt A. Protocol deviation and violation. Perspect Clin Res. 2012;3(3):117. doi:10.4103/2229-3485.100663.View ArticlePubMedPubMed CentralGoogle Scholar