Estimation of treatment preference effects in clinical trials when some participants are indifferent to treatment choice
 Stephen D. Walter^{1}Email author,
 Robin M. Turner^{2},
 Petra Macaskill^{3},
 Kirsten J. McCaffery^{3} and
 Les Irwig^{3}
DOI: 10.1186/s128740170304x
© The Author(s). 2017
Received: 16 August 2016
Accepted: 2 February 2017
Published: 20 February 2017
Abstract
Background
In the twostage randomised trial design, a randomly sampled subset of study participants are permitted to choose their own treatment, while the remaining participants are randomised to treatment in the usual way. Appropriate analysis of the data from both arms of the study allows investigators to estimate the impact on study outcomes of treatment preferences that patients may have, in addition to evaluating the usual direct effect of treatment. In earlier work, we showed how to optimise this design by making a suitable choice of the proportion of participants who should be assigned to the choice arm of the trial. However, we ignored the possibility of some participants being indifferent to the treatments under study. In this paper, we extend our earlier work to consider the analysis of twostage randomised trials when some participants have no treatment preference, even if they are assigned to the choice arm and allowed to choose.
Methods
We compare alternative characterisations of the response profiles of the indifferent or undecided participants, and derive estimates of the treatment and preference effects on study outcomes. We also present corresponding test statistics for these parameters. The methods are illustrated with data from a clinical trial contrasting medical and surgical interventions.
Results
Expressions are obtained to estimate and test the impact of treatment choices on study outcomes, as well as the impact of the actual treatment received. Contrasts are defined between patients with stated treatment preferences and those with no preference. Alternative assumptions concerning the outcomes of undecided participants are described, and an approach leading to unbiased estimation and testing is identified.
Conclusions
Use of the twostage design can provide important insights into determinants of study outcomes that are not identifiable with other designs. The design can remain attractive even in the presence of participants with no stated treatment preference.
Keywords
Randomised trials Design Estimation Participant preferencesBackground
Clinical trial outcomes may be affected by preferences that trial participants might have between the treatments under comparison. Preference effects can be substantial [1], but they are unobservable in standard trial designs; however, they are estimable in the twostage randomised trial, in addition to the usual treatment effect [2, 3]. Using this design, a randomly sampled subset of patients are permitted to choose their own treatment, with the remainder randomised to treatment in the usual way. Estimated preference effects often provide additional insight into determinants of the study response, beyond that of the treatment effect alone [1–4]. As we have reviewed elsewhere [1], several preferencebased designs have been employed in biomedical research in recent years [5–18].
We previously showed how to optimise the twostage design by controlling the proportion of patients who are allocated to the choice arm (and hence can choose their treatment) versus the random arm (where they are randomised to treatment) [3]. In our previous illustrative example, involving the management of women with abnormal cervical screening results [4, 5], the investigators compared a novel intervention with standard care, which effectively defined a treatment preference for all participants. However, in other situations, some participants may have no treatment preference, despite being informed about the alternatives. For instance, in our further example discussed later [6, 7], 69% of participants in the choice arm had no preferred treatment.
In general, patients may be unable or unwilling to choose a treatment because they want the doctor to decide, or through anxiety or a lack of information or decisionmaking confidence. Note specifically that “no preference” is the observed preference value obtained after due process of presenting treatment options to the patient, but it is not “missing” data.
No previous paper has discussed how to deal with patients who are indifferent to treatment choice, i.e., who are “undecided”, in more than passing fashion. Apart from our own previous work [1], parameter estimation in the twostage design has not been described. The possibility of undecided patients was mentioned when the design was originally proposed by Rucker [2], but only concerning significance testing. Rucker assumed that such participants either did not exist, or that preference effects did not apply to them. We show here that these approaches result in potentially biased parameter estimates, but also that alternative (and testable) assumptions are possible. We believe this issue is an important gap in the existing literature, on a topic with considerable practical applicability.
Our previous work assumed that all patients had a treatment preference, but we now extend our methods to allow for undecided patients. We obtain estimates, variances and test statistics for various preference effects, and estimate the usual treatment effect in this design. We provide a general characterisation of the undecided group, including alternative assumptions about their response profile, and we determine if these assumptions are empirically testable.
We illustrate the calculations using actual trial data, discuss alternative ways that data from undecided participants can be used, and compare our analysis to that of the original trial investigators. In our Discussion, we consider the appeal of the twostage design when participants with no treatment preference occur, including their impact on study feasibility and efficiency.
Methods
Observations and parameters by group and actual treatment for the 2stage randomised design
Actual treatment  Choice arm (Total sample size = m)  Random arm (Total sample size = n)  

Choose A  Choose B  Undecided (and randomised)  
A  Sample size  m _{ 1 }  —  m _{ 3 } /2  n _{ 1 } 
Observations  x _{ 11 } ,…x _{ 1,m1 }  v _{ 11 } ,…v _{ 1,m3/2 }  y _{ 11 } ,… y _{ 1,n1 }  
Mean  μ _{ 11 }  (μ _{ 12 } )  μ _{ 13 }  μ _{ 1 }  
SD  σ _{11}  σ _{13}  σ _{1}  
B  Sample size  —  m _{ 2 }  m _{ 3 } /2  n _{ 2 } 
Observation  x _{ 21 } ,…x _{ 2,m2 }  v _{ 21 } ,…v _{ 2,m3/2 }  y _{ 21 } ,…y _{ 2,n2 }  
Mean  (μ _{ 21 } )  μ _{ 22 }  μ _{ 23 }  μ _{ 2 }  
SD  σ _{22}  σ _{23}  σ _{2} 
Ideally, we would like to observe \( {\mu}_{ij} \) as the mean response for participants in preference group j who receive treatment i, for every (i, j) combination, but only four of the six possible subgroups are observable. We can observe participants in the choice arm who have a treatment preference, with expected outcomes \( {\mu}_{11} \) and \( {\mu}_{22} \) for participants who prefer and receive treatments A or B, respectively. Because undecided participants are rerandomised, we can estimate \( {\mu}_{13} \) and \( {\mu}_{23} \) from the choice arm. The remaining two subgroups, corresponding to \( {\mu}_{12} \) and \( {\mu}_{21} \), are unobservable, because all participants in the choice arm with a treatment preference will receive that treatment.
Among the m participants in the choice group, there are m _{ 1 } and m _{ 2 } who choose treatments A and B, respectively. Among the remaining participants with no preference, we assume that m _{ 3 }/2 are randomised to each of A and B. In the random arm, there is only one observed group for each treatment, containing unknown mixtures of treatment preferences. Here we observe samples of size n _{ 1 } and n _{ 2 } under treatments A and B.
Because of the potential effects of treatment preferences, we take a general approach and allow for different expected outcomes for the corresponding treatment groups in the choice and random arms, and for indifferent participants to have different expected outcomes from the other observable groups. We comment later on the implications of alternative assumptions concerning the responses in the indifferent participants.
We define the direct treatment effect by Δτ = τ _{1} − τ _{2}, or equivalently μ _{1} − μ _{2}. One can estimate Δτ from the random arm, through the sample estimate \( {\overline{Y}}_1 \) – \( {\overline{Y}}_2 \) of μ _{ 1 } – μ _{ 2.} By virtue of the randomisation, this estimate will be unbiased, because participant preferences will be balanced in expectation. However, the estimate will be less precise than in a parallel group trial of the same total size.
We now consider the selection and preference effects. Because there are three preference groups, there are two possible contrasts for each effect.
Selection effect
Preference effect
Again, this devolves to a simpler previous result when there are no undecided participants ([3], Eq. 3).
Second contrasts for selection and preference effects
Significance tests and confidence intervals for selection and preference effects
We can use variance (9) to establish a confidence interval for Δν.
Similar derivations are possible for testing and estimating the second contrasts Δν′ and Δπ′; for brevity these are not shown here, but some simplified results are given below.
The results derived thus far are completely general in allowing for different variances in the various study subgroups, and for sample variation in the preference distribution within the choice arm. Considerable simplification occurs if these conditions are relaxed. First, we may assume that the outcome variance is constant. Second, we may ignore sample variability in the preference distribution in the choice arm. We refer to the variances as unconditional when sample variability in preferences is taken into account (as in Eqs. 7, 8, 9, 10 and 11), and conditional when the preference distribution is taken as fixed.
Possible assumptions for responses in undecided participants
In the original development of the twostage design [2], Rucker proposed that one should either assume that there were no undecided participants (γ = 0), or that these people were not subject to selection or preference effects (μ _{ i3} = μ _{ i } for i = 1, 2, respectively). Because the {μ _{ i3}} are estimable from undecided participants within the choice arm, one can test the null hypotheses H0: μ _{13} = μ _{1} and H0: μ _{23} = μ _{2} by comparing the outcomes of the undecideds to their similarly treated counterparts in the random arm. When testing selection and preference effects, these assumptions are equivalent to assuming that e _{ 1 } = e _{ 2 } = 0; however, these terms appear in expressions such as (7) and (9, 10 and 11), implying that ignoring them can bias the test statistics or confidence intervals of these parameters.
An alternative approach is to assume that μ _{ ii } = μ _{ i3} for i = 1, 2, or that the expected outcomes are the same in participants with or without a preference, for each treatment. This amounts to the socalled “exclusion restriction” whereby the expected outcome is determined only by the treatment actually received. Again these assumptions are testable in the twostage design, using data from the choice arm.
Another strategy would be to exclude undecided participants from the analysis. This approach would make it appear (incorrectly) that all participants in the random arm have a preferred treatment.
One can show that expressions (4) and (5) give consistent estimators of the selection and preference effects even in the presence of indifferent participants. Conversely if either set of assumptions above is adopted, but is actually false, biased estimation will occur. The bias will be important unless the proportion of indifferent participants is small.
Results
To illustrate these calculations, we will use data from a trial to compare alternative interventions for heavy menstrual bleeding [6, 7]. The treatment options were surgery or medical management without surgery. Women were randomised to a “conventional trial” subgroup (random arm) or to a “patient preference” subgroup (choice arm). In the choice arm, women were asked if they had a preference for one of the treatments, and if so, they could have it.
There were 227 women with outcome data available, of whom 130 (57%) were assigned to the choice arm. Within the choice arm, 19 elected to take medical treatment, and 21 decided to have surgery. The remaining 90 women (69%) had no preference, and were randomised (45 to each treatment). Thus while the rate of undecided participants was high, the two treatments were selected in somewhat similar numbers by women who were able to make a choice.
Summary of results from a twostage randomised trial of medical vs. surgical treatment
Actual treatment  Choice arm (Total sample size = m)  Random arm (Total sample size = n)  

Chose medical  Chose surgical  Undecided (and randomised)  
Medical  Sample size  19  45  49  
Mean  16.6  18.4  17.2  
SD  8.7  10.7  5.2  
Surgical  Sample size  21  45  48  
Mean  5.9  4.3  5.1  
SD  7.2  5.2  7.7 
The investigators noted some baseline similarities between those choosing or being randomised to treatment, and they therefore aggregated the results from the randomised women in the random and choice arms, and reported outcomes only according to the actual treatment. In this paper we are able to use more detailed data on preferences within the choice arm (kindly provided by the investigators), allowing us to structure the results in the format of our Tables 1 and 2.
The direct effect of treatment (estimated from the random arm), was 12.10, with a standard error of 1.54 based on a standard deviation pooled over all the data. The 95% confidence interval was (9.1, 15.1), indicating a significant treatment effect in the direction of better outcomes for women having surgery.
The estimated first selection effect contrast \( \varDelta \widehat{\nu} \) was 3.03; using the unconditional approach and allowing for different subgroup variances, with standard error 6.72. Testing this effect gives T = 18.6 (SD = 40.8); z = 0.46 (p = 0.65, 2sided test). The conditional standard error of \( \varDelta \widehat{\nu} \) when the outcome variance was taken as constant across study groups was 6.64, only fractionally smaller than its unconditional counterpart. For simplicity, conditional standard errors were used in all further analyses.
The second selection contrast \( \varDelta {\widehat{\nu}}^{\prime } \) was 0.57, with conditional standard error 3.62. While there is no strong evidence of selection effects, the first contrast indicates better outcomes for women who would select surgery (treatment B) if allowed to do so, this being in addition to the direct benefit of surgery.
The two estimated preference effects are \( \varDelta \widehat{\pi} \) = 0.93 and \( \varDelta \widehat{\pi^{\prime }} \) = − 0.49 (standard error = 6.64 for both). The first contrast, being positive (although not significant), indicates that the medical versus surgical effect is greater for women who preferred medical treatment.
We can also empirically test the alternative assumptions for the outcomes among the undecideds. First, using Rucker’s approach [2], we can compare the outcomes for women on medical treatment, between the undecided participants and those in the random arm: this gives a mean difference of 18.40–17.20 = 1.20 (z = 0.77, p = 0.44). Similar analysis of the corresponding groups of women on surgery gives z = − 0.51 (p = 0.61). Second, an assessment of the “exclusion restriction” assumption leads to similar findings, with only small differences in outcomes between women who chose their treatment in the choice arm, or were assigned to it in the random arm. Hence, in this example, there is no compelling evidence of systematically different outcomes for undecided women. However, we feel it is still advantageous to allow for the existence of undecided participants, in order to avoid bias. As noted earlier, we do not recommend simply ignoring the data from undecided participants.
Discussion
We have extended the methodology for twostage randomised trials to allow for participants who do not have a preferred treatment. As noted earlier, the twostage design has been used in a variety of settings, and in some cases the selection and preference effects have been important, even when lacking a large treatment effect. From the perspective of shared decisionmaking concerning treatment options, it would be important to know about these effects.
Investigators who consider adopting the twostage trial design would presumably hope that most participants have a preferred treatment. For instance, trials using patient decision aids try to improve patient understanding of the treatment options and reduce the number of patients who are uncertain about their treatment, to help them make an informed choice reflecting their own values [19]. For patients who are unable or unwilling to choose, the decision will likely revert to the clinician proposing a treatment, and obtaining the tacit consent of the patient.
Despite the complication of undecided participants, the twostage design remains attractive to people who do have a preferred treatment, and it still provides important information that is unobtainable from a conventional design. Preference effects may enhance the direct effect of treatment (as in our example), or act in the opposite direction. One may also compare the responses of decided vs. undecided individuals, as this may also be predictive of better or worse outcomes; these comparisons are accessible through our defined second contrasts of the selection and preference effects. Depending on the context, it seems better to allow study participants the option of declaring no preference, rather than pressuring them into a “forced” choice of treatment; in this scenario, “no preference” is a valid observation, and not “missing”. However, the question of undecided patients may not arise in some circumstances, for instance where one treatment is usual care (e.g., [4, 5]).
We remarked earlier that it is inappropriate to simply ignore undecided participants in the choice arm, because of the potential for bias. Interestingly, in our example study [6, 7], the investigators elected to include the undecided participants, but they aggregated them with similarly treated participants in the random arm. Because all patients in this analysis are randomised, the estimated treatment effect is unbiased, but it lacks generalizability because participants with a declared preference are excluded.
Despite the advantages of the twostage design, it must be recognised that the existence of undecided or indifferent patients may reduce its appeal. First, if a large proportion of participants cannot decide on treatment, estimating all the effects of interest (treatment, selection and preference effects) becomes less precise. Second, the presence of undecided participants leads to greater analytic complexity. Finally, the burden of explaining the study to participants may become unattractive if relatively few people will actually be able to make a treatment choice. However, the investigators in our example found that using this design “did not affect recruitment” compared to conventional approaches [6, 7]. Nevertheless, it should be noted that despite this encouraging finding, outcomes may still differ between participants in the random arm and the subgroup who are randomised within the choice arm.
In our example [6, 7], there was a high proportion of undecided participants, with less than one third of the patients in the choice arm declaring a preference. In this study, the information sheet for women in the choice arm contained only two additional sentences (“We do realise that some women may have a preference for one of the treatments. If you feel strongly that you want one treatment in particular, then please tell us”) compared to the information provided in the random arm. The mention of “strong” preferences may have deterred some women from choosing a treatment for which they had only a weaker preference. Researchers carrying out preference trials should consider carefully the methods they use to elicit preferences and their impact on the preferences that participants feel able to express.
Participants with no preference were also documented in another twostage design [9], a trial comparing medication and cognitive behavioural therapy for the treatment of depression. Preferences were recorded on a 5point scale, with participants (19%) who chose the middle value being defined as having “no clear treatment preference” or “indecisive”. In other examples of the twostage design [8, 16–18, 20], no undecided participants were reported, but it is unclear if such individuals were simply ignored, or if initially undecided persons were somehow persuaded to make a definitive choice before continuing into the study.
Further work is required to extend our earlier results [3] on optimising study efficiency while allowing for undecided participants. Optima here will depend on the proportion of participants who can decide on a treatment, as well as factors such as the preference rates for the two treatments among the decided individuals. The preference distribution will also affect the power to detect selection and preference effects [21].
We limited our attention to trials with equal numbers of participants being assigned to each treatment group in the random arm. Another research topic of future interest will therefore be to extend our methods to allow for unequal treatment group sizes. Unequal assignments might be desirable for ethical reasons, for instance if there is a large majority preference for one treatment, or if one treatment has a larger response variance. In such cases, the investigators might wish to to improve study efficiency, by striving for overall balance in the trial, for example. We have discussed the ethics of unequal randomisation in somewhat more detail elsewhere [1].
We adopted a model in which preferences were assumed to have fixed effects. This was to conform to Rucker’s original development, and indeed to our own earlier work in this area, which we have extended here. A further interesting extension would be to allow for preferences (and, potentially, the treatment) to have random effects. We note that all our estimates and tests are based on large sample normal approximations, and further work would be needed to consider small sample situations. One could, for instance, replace our zstatistics with tstatistics. Our previous work on sample size and power for the twostage design [21] showed that particularly small studies will be inadequate to reliably estimate the parameters of interest. Rucker’s investigation of empirical type I error rates showed excellent adherence to nominal type I error rates with as few as 20 patients in each of the random and choice arms [2]. These results suggest that the large sample approximations described here will be adequate for most practical problems.
A reviewer pointed out that participants in the random arm may be randomised to a treatment that they do not like, and this in turn may influence outcomes. This is a valid point, but one that also affects a number of other designs, including standard parallel group trials, in exactly the same way. It has previously been noted that parallel group trials involve arms that may not be comparable after randomisation, because some patients will be pleased to get their preferred treatment, while others will be disappointed (this assuming that participants are not blinded) [22]. This phenomenon has been termed “reluctant acquiescence” [23], and it may in turn affect the estimated treatment effect. To the extent this is a problem for the twostage design, exactly the same problem exists in parallel group designs, in terms of the impact on the estimated treatment effect. However, note that because the estimates of the selection and preference effects in a twostage design are based partly on the data from the random arm, there may also be subsidiary implications for those estimates as well.
There are related issues in a variety of other preferencebased designs, as we have reviewed elsewhere [1]. For example, there are challenges in how to present information about treatments and then eliciting preferences reliably, and one must recognise that there are potential differences between what are stated as the preferences for treatment in a general sense, and actual treatments chosen when participants are allowed to choose.
As we have reviewed elsewhere [1], randomised trial designs can allow for patient preference effects in various ways. There is also some literature on the impact of clinician preferences for treatments, and the associated “expertise bias” [24, 25]. The various designs that accommodate patient or clinician preferences are differentiated by whose preference is ascertained, how and when they are measured, and if treatment assignments take those preferences into account. Among these designs, the twostage approach is unique in being able to estimate selection and preference effects in addition to the direct treatment effects [26, 27]. It remains a candidate design even when the complication of indifferent participants arises. The results in this paper have shown how the information from such participants can be used in the analysis for twostage trials, to further elucidate the impact of treatment preferences on study outcomes.
Conclusions
Treatment preferences among clinical trial participants are potentially important determinants of study outcomes, and we have demonstrated how data from twostage randomised trials can be analysed to estimate such effects. We have extended our earlier work to now allow for the possibility of participants who are indifferent or undecided about a preferred treatment, and we discussed alternative possible assumptions about their response patterns, compared to participants with a declared treatment preference. We also have shown how to obtain unbiased estimates and tests of the preference effects in the trial as a whole. Despite the presence of indifferent or undecided participants, the two stage design is nevertheless an attractive option for the examination of preference effects, a feature which is not possible with conventional trial designs.
Abbreviations
 SD:

Standard deviation
Declarations
Acknowledgements
We thank Dr KG Cooper for provision of data from his study, used in our illustrative example.
Funding
This work was partially supported by funds from the Natural Sciences and Engineering Research Council.
Availability of data and materials
All data generated or analysed during this study are included within this published article. The study cited in references 6 and 7 provides the basis for our illustrative example. Dr Kevin Cooper (Consultant Gynaecologist, Aberdeen Royal Infirmary, Scotland) kindly provided additional details of the study that were not shown in those references, and gave us permission to use them in our published work. Specifically, these details were the provision of summary statistics (sample sizes, means, and standard deviations) of the study outcome variable, according to the randomly assigned arm of the study, and according to the stated treatment preferences in the choice arm. These summary statistics are shown within the current paper.
Authors’ contributions
All authors read and approved the final version of the manuscript. Their specific contributions are as follows: SW: provided the overall leadership for this work, and was primarily responsible for developing the analytic method and for data analysis; wrote the initial and final drafts of the paper. RT: contributed to the conception and design of the study, revised the manuscript for important intellectual content, and agrees to be accountable for all aspects of the work. PM: made substantial contributions to the conception and design of the study, particularly interpretation and derivation of equations; was involved in revising the manuscript for important intellectual content, and takes public responsibility for the content; agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. KM: commented on multiple drafts of the paper and made intellectual contribution to its content, and agrees to be accountable for the work. LI: was involved in revising the manuscript for important intellectual content; takes public responsibility for the content; and agrees to be accountable for the work.
Authors’ information
None.
Competing interests
The authors declare that they have no competing interests.
Consent for publication
Not applicable.
Ethics approval and consent to participate
Not applicable.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Authors’ Affiliations
References
 Walter SD, Turner R, Macaskill P, McCaffery K, Irwig L. Beyond the treatment effect: evaluating the effects of patient preferences in randomised trials. Stat Methods Med Res. 2014. doi:10.1177/0962280214550516.Google Scholar
 Rucker G. A twostage trial design for testing treatment, selfselection and treatment preference effects. Stat Med. 1989;8:477–85.View ArticlePubMedGoogle Scholar
 Walter SD, Turner RM, Macaskill P, McCaffery K, Irwig L. Optimal allocation of participants for the estimation of selection, preference and treatment effects in the twostage randomised trial design. Stat Med. 2012;31:1307–22.View ArticlePubMedGoogle Scholar
 McCaffery KJ, Turner R, Macaskill P, Walter SD, Chan SF, Irwig L. Determining the impact of informed choice: separating treatment effects from the effects of choice and selection in randomised trials. Med Decis Making. 2011;31:229–36.View ArticlePubMedGoogle Scholar
 McCaffery KJ, Irwig L, Turner R, Chan SF, Macaskill P, Lewicka M, Clarke J, Weisberg E, Barratt A. Psychosocial outcomes of three triage methods for the management of borderline abnormal cervical smears: an open randomised trial. BMJ. 2010;340:b4491.View ArticlePubMedPubMed CentralGoogle Scholar
 Cooper KG, Grant AM, Garratt AM. The impact of using a partially randomised patient preference design when evaluating alternative managements for heavy menstrual bleeding. Br J Obstet Gynaecol. 1997;104:1367–73.View ArticlePubMedGoogle Scholar
 Cooper KG, Parkin DE, Garratt AM, Grant AM. Twoyear follow up of women randomised to medical management or transcervical resection of the endometrium for heavy menstrual loss: clinical and quality of life outcomes. Br J Obstet Gynaecol. 1999;106:258–65.View ArticlePubMedGoogle Scholar
 Floyd AHL, Moyer A. Effects of participant preferences in unblinded randomized controlled trials. J Empir Res Hum Res Ethics. 2010;8:81–93.
 Mergl R, Henkel V, Allgaier AK, Kramer D, Hautzinger M, Kohnen R, Coyne J, Hegerl U. Are treatment preferences relevant in response to serotonergic antidepressants and cognitivebehavioural therapy in depressed primary care patients? Results from a randomized controlled trial including a patients’ choice arm. Psychother Psychosom. 2011;80:39–47.View ArticlePubMedGoogle Scholar
 Rokke PD, Tomhave JA, Jocic Z. The role of client choice and target selection in selfmanagement therapy for depression in older adults. Psychol Aging. 1999;14:155–69.View ArticlePubMedGoogle Scholar
 Long Q, Little RJ, Lin X. Causal inference in hybrid intervention trials involving treatment choice. J Am Stat Assoc. 2008;103:474–84.View ArticleGoogle Scholar
 Janevic MR, Janz NK, Dodge JA, Lin X, Pan W, Sinco BR, Clark NM. The role of choice in health education intervention trials: a review and case study. Soc Sci Med. 2003;56:1581–94.View ArticlePubMedGoogle Scholar
 Rose JP, Geers AL, Rasinski HM, Fowler SL. Choice and placebo expectation effects in the context of pain analgesia. J Behav Med. 2012;35:462–70.View ArticlePubMedGoogle Scholar
 Kernahan C, Bettencourt BA. Motivated processing of black and white targets: the situation of choice versus assignment. Basic Appl Soc Psych. 2002;24:125–44.View ArticleGoogle Scholar
 Noel PH, Larme AC, Meyer J, Marsh G, Correa A, Pugh JA. Patient choice in diabetes education curriculum. Nutritional versus standard content for type 2 diabetes. Diabetes Care. 1998;21:896–901.View ArticlePubMedGoogle Scholar
 Burke LE, Styn MA, Steenkiste AR, Music E, Warziski M, Choo J. A randomized clinical trial testing treatment preference and two dietary options in behavioural weight management: preliminary results of the impact of diet at 6 months  PREFER study. Obesity. 2006;14:2007–17.View ArticlePubMedGoogle Scholar
 Burke LE, Hudson AG, Warziski MT, Styn MA, Music E, Elci OU, Sereika SM. Effects of a vegetarian diet and treatment preference on biochemical and dietary variables in overweight and obese adults: a randomized clinical trial. Am J Clin Nutr. 2007;86:588–96.PubMedGoogle Scholar
 Burke LE, Warziski MT, Styn MA, Music E, Hudson AG, Sereika SM. A randomized clinical trial of a standard versus vegetarian diet for weight loss: the impact of treatment preference. Int J Obes (Lond). 2008;32:166–76.View ArticleGoogle Scholar
 Stacey D, Légaré F, Col NF, Bennett CL, Barry MJ, Col NF, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2014. doi:10.1002/14651858.Google Scholar
 Eisenberg DM, Post DE, Davis RB, Connelly MT, Legedza AT, Hrbek A, et al. Addition of choice of complementary therapies to usual care for acute low back pain  a randomized controlled trial. Spine. 2007;32:151–8.View ArticlePubMedGoogle Scholar
 Turner RM, Walter SD, Macaskill P, McCaffery KJ, Irwig L. Sample size and power when designing a randomised trial for the estimation of treatment, selection and preference effects. Med Decis Making. 2014;34:711–9.View ArticlePubMedGoogle Scholar
 McPherson K, Chalmers I. Incorporating patient preferences into clinical trials. BMJ. 1998;317:78.View ArticlePubMedPubMed CentralGoogle Scholar
 Tilbrook H. Patients’ preferences within randomised trials: systematic review and patientlevel metaanalysis. BMJ. 2008;337:a1864.View ArticleGoogle Scholar
 Devereaux PJ, Bhandari M, Clarke M, Montori VM, Cook DJ, Yusuf S, et al. Need for expertisebased randomized controlled trials. BMJ. 2005;330:88–91.View ArticlePubMedPubMed CentralGoogle Scholar
 Walter SD, Ismaila AS, Devereaux PJ. for the SPRINT study investigators (2008). Statistical issues in the design and analysis of expertisebased randomized clinical trials. Stat Med. 2008;27:6583–96.View ArticlePubMedPubMed CentralGoogle Scholar
 Goodman LA. On the exact variance of products. JASA. 1960;55:708–13.View ArticleGoogle Scholar
 Johnson NL, Kotz S, Balakrishnan N. Discrete multivariate distributions. New York: John Wiley & Sons; 1997.Google Scholar