- Research article
- Open Access
- Published:
Questions asked and answered in pilot and feasibility randomized controlled trials
BMC Medical Research Methodology volume 11, Article number: 117 (2011)
Abstract
Background
In the last decade several authors have reviewed the features of pilot and feasibility studies and advised on the issues that should be addressed within them. We extend this literature by examining published pilot/feasibility trials that incorporate random allocation, examining their stated objectives, results presented and conclusions drawn, and comparing drug and non-drug trials.
Methods
A search of EMBASE and MEDLINE databases for 2000 to 2009 revealed 3652 papers that met our search criteria. A random sample of 50 was selected for detailed review.
Results
Most of the papers focused on efficacy: those reporting drug trials additionally addressed safety/toxicity; while those reporting non-drug trials additionally addressed methodological issues. In only 56% (95% confidence intervals 41% to 70%) were methodological issues discussed in substantial depth, 18% (95% confidence interval 9% to 30%) discussed future trials and only 12% (95% confidence interval 5% to 24%) of authors were actually conducting one.
Conclusions
Despite recent advice on topics that can appropriately be described as pilot or feasibility studies the large majority of recently published papers where authors have described their trial as a pilot or addressing feasibility do not primarily address methodological issues preparatory to planning a subsequent study, and this is particularly so for papers reporting drug trials. Many journals remain willing to accept the pilot/feasibility designation for a trial, possibly as an indication of inconclusive results or lack of adequate sample size.
Background
In the last decade a number of authors have reviewed the justification for describing a trial as a pilot or feasibility study in terms of its content and the questions it addresses [1–5]. Lancaster et al [2] list as legitimate objectives of a pilot study: sample size calculation; providing a dummy run of trial procedures/the protocol; testing data collection forms or questionnaires; testing how randomization procedures work; determining recruitment and consent rates; examining the acceptability of the intervention; and selection of the most appropriate primary outcome measure. Thabane et al [5] categorize reasons to conduct a pilot study into four groups: assessing the feasibility of processes that are key to the success of the main study; assessing time and resource problems; potential human and data management problems; and scientific issues including the assessment of treatment safety, dose, response effects and variance of the effect. They also include a checklist of items to include in reports of pilot studies. Arain et al [4] recommend the NIHR Evaluation, Trials and Studies Coordinating Centre definitions [6] which describe a pilot study as a miniature version of a main study run to test whether components of the main study work together, while feasibility studies are pieces of research done before a main study to answer the question "Can this study be done?". According to these definitions both pilot and feasibility studies play a preliminary role in the design stage of a subsequent larger trial, and do not themselves address efficacy.
Arain et al [4] comment that researchers applying for funding for trials inadequately powered to address clinically meaningful hypotheses may adopt the designation of a pilot study in the hope of a more favourable review. When studies are prepared for publication authors may similarly believe that labeling a small trial as a pilot increases its chance of acceptance. Although Arain et al found that the editors of five high ranking medical journals did not encourage publication of pilot studies because of their perceived lack of rigour, it is possible that other journals are more accommodating. In an editorial in the journal Circulation, Loscalzo [7] proposed a binary classification: trials designated a priori as pilots, and those redefined a posteriori. During a five year period 41 pilot trials were published in the journal. Many had been designated as pilots at the request of the editorial office to alert readers to uncertainty in the generalizability of their results and their preliminary and exploratory nature. Such a policy is likely to result in trials primarily addressing efficacy being described as pilots, contrary to the NIHR and other recent definitions.
In comparison to the well established pathways of development for pharmaceuticals, prior to 2000 there was little specific guidance on the development of procedures involved in trials of non-pharmacological interventions, possibly because of their complex and heterogeneous nature. The MRC guidelines for the evaluation of complex interventions published in 2000 [8] and revised in 2009 [9] emphasize the importance of testing procedures before planning an evaluation, and also the circular nature of development, feasibility and piloting, evaluation, implementation, and further development. There is no specific guidance for non-pharmacological interventions that do not meet the MRC definition of complexity. In this paper we review a random sample of 50 papers reporting randomized controlled trials (RCTs) published in journals covered by the MEDLINE and EMBASE databases where authors described their trial as a 'pilot' or addressing 'feasibility' in the title. We review papers published between 2000 and 2009, the ten years following the publication of the first MRC guidelines on complex interventions. We examine stated objectives, results and conclusions drawn, and in particular whether these relate to methodological issues, efficacy or safety/toxicity: comparisons are drawn between papers reporting drug and non-drug trials.
Methods
We searched the EMBASE and MEDLINE databases on 29th July 2010 to identify papers reporting parallel group trials with one or both of the words 'Pilot' and 'Feasibility' in the title. To be included, papers had to be published between 2000 and 2009, written in English, studying humans and indexed as an RCT. Using computer generated random numbers we selected 50 of those identified for full review [10–59]. The sample size was chosen taking into account resources available and the detailed review required, it allows percentages between 10% and 17.5% to be estimated with a 95% confidence interval (CI) of ± 10%. The search was repeated on the 21st February 2011 to gain a more complete estimate of the number of relevant papers in 2009: two papers were selected from those identified on this date to replace papers found to be ineligible at a late stage.
A form was designed to document characteristics of the selected papers. It was tested by all three authors on three randomly selected papers which were not part of the main review sample. Minor modifications were made after the first two papers of the main sample had been reviewed. We defined drug trials as those involving the administration of a discrete chemical entity, substance, or biological agent by mouth or other route, for example by injection. Questions on the form related to: any blinding and in particular double blinding; the numbers of active and placebo/control arms; whether multiple centres were involved (sometimes deduced from the number of institutional review boards mentioned); the target and actual sample sizes; and any justification given for the target sample size.
Research objectives stated in the Abstract and Introduction sections were coded as relating to methodological issues, efficacy, or safety/toxicity, as were statements summarizing results and conclusions chosen for inclusion in the Abstract. Objectives had to be explicitly stated, it was not enough for the reviewer to deduce what the objectives might have been from results presented, or conclusions drawn. Efficacy conclusions in the Abstract were rated as indicating that the experimental intervention had not been shown to have benefit or had no benefit, that it showed promise, or that it showed actual benefit. The accuracy of conclusions drawn by the authors was not verified. An example of an efficacy conclusion in the Abstract rated as indicating the intervention had not been shown to have benefit or had no benefit is
"With the numbers studied, we failed to find a significant difference between the two groups; thus we have no evidence of a benefit from botulinum toxin injection in the treatment of chronic tennis elbow" [34];
one rated as indicating the intervention to have promise is
"Therefore, the colonic colplasty seems to be an attractive pouch design because of its feasibility, simplicity, and effectiveness." [31];
and one rated as indicating the intervention to have actual benefit is
"Sleep educational programs for secondary students are recommended to improve information about sleep." [23].
Numerical results presented in the Results sections were classified as relating to methodological issues, efficacy, or safety/toxicity. We looked for counts of trial participants experiencing methodological problems or side effects, or numerical summaries of statistical findings: text statements that a procedure was feasible for example were not enough to qualify as a result. Methodological results were recorded separately for: recruitment, retention, compliance/adherence to intervention, blinding procedures, acceptability of the intervention to participants, other aspects of the intervention, outcome assessment, logistics of the randomization procedure, acceptability of trial procedures, or the logistics of multi-centre procedures. Selection of these topics was in part based on the issues that Lancaster et al [2] list as constituting pilot studies. We checked the Methods sections to see whether methodological results were reported there, sometimes they formed part of a CONSORT flowchart [60] for example. Depth of coverage was coded as none, brief, detailed or tabulated/graphical presentation. Significance tests and CIs presented for efficacy outcomes were examined to see whether they indicated between or within group significant differences, but no attempt was made to judge which findings were the primary evaluation of efficacy.
In the Discussion sections we again rated coverage of methodological issues, efficacy, and safety/toxicity as none, minimal, substantial, or the major focus of the section. An example of discussion concerning methodological issues rated as minimal was
"The results are promising but not conclusive because of the low numbers of patients studied, and we recommend that a sufficiently powered study should be performed." [29];
and the following example was rated as substantial
"In order to show an assumed clinically relevant difference of 2 kg, with an 80% power and a type-I error of 5%, 300 patients would be needed (150 in each treatment group). If a subsequent study were to be planned, it would be advisable to use the mean change in grip strength as a primary variable because the variability for this parameter was lowest in the present study and it came close to identifying a significant difference between groups (p = 0.196). In addition, grip strength is a quantifiable measurement of effect, unlike the more subjective measurement of pain." [34].
The NIHR definitions indicate that pilot and feasibility studies should be preliminary research prior to a main study: we were therefore interested in whether authors stated they were conducting a further trial (or were scheduled to start one in the near future). If this was mentioned it was usually in the Discussion section. Since the above two quotes were the only mention of future trials in each paper we did not consider either set of authors to be actually conducting a future trial. Comments in the Discussion concerning lack of power or small sample size were noted.
Finally we recorded whether Conclusions sections contained statements concerning methodological issues, efficacy, or safety/toxicity. The Conclusion section could be a specifically labelled section, a paragraph of the Discussion clearly listing conclusions, or presented as a box: where there was no such section missing was coded not the absence of a relevant conclusion.
The 50 papers were assessed by MS and difficulties arising were discussed with RMP and MW. Blyth-Still-Casella 95% CIs for single percentages and exact CIs for Rate Ratios (RR) were obtained in StatXact [61]. Ordinal ratings were compared between groups in Mann-Whitney U tests, and percentages in exact Pearson's chi-squared tests.
Results
After removal of duplicates our EMBASE/MEDLINE search identified 3,581 papers (see Table 1). In order to achieve a sample of 50 suitable papers a further 25 were rejected for the reasons shown in Table 1. The final two papers were excluded at a late stage because the words 'pilot' or 'feasibility' in the title did not relate to the trial (in one 'pilot' was part of the name of the intervention and in the other the intervention aimed to increase the feasibility of a further procedure). When the search was repeated on 21st February 2011 the number of papers had increased to 3652 (Figure 1). The frequency of papers rose steeply with time. Although not formally evaluated it is likely that, as in the sample of papers selected for review, a third would not meet our eligibility criteria. The majority (3120, 85%) of papers had the word 'pilot' in the title; 479 (13%) had the word 'feasibility'; and 50 (1%) had both.
Frequencies of papers identified with 'pilot', 'feasibility' or both in the title (searched on 21 st February 2011).
Table 2 describes the characteristics of the trials reported in the papers selected for review [10 - 59] The percentages with 'pilot' or 'feasibility' in the title were similar to those amongst the 3,652 papers identified. In four titles [32, 41, 45, 46] the word 'feasibility' described the trial: in the other four [30, 38, 42, 43] it described the intervention. Over half of the trials, (28, 56%, CI 41% to 70%) evaluated drugs. The majority (29, 58%) were single centre trials, 19 (38%) were multi-centre, and in two cases we were unable to determine whether one or more centres were involved. Most trials consisted of one active and one control arm, but the drug trials often had active arms at several doses and 8/28 (29%) had no placebo arm. One non-drug trial was unusual because it had 12 arms. It was a factorial trial carried out over the internet with one factor being six ways of presenting treatment effects to participants crossed with a second factor being the order of eliciting their understanding of treatment consequences from visual analogue or category rating scales [20]. The drug trials were more likely than non-drug trials to incorporate blinding (75% vs 32%, RR 2.4, CI 1.3 to 4.7), or to be described as double blinded (54% vs 5%, RR 11.8, CI 2.3 to 167.6). Most papers didn't justify the sample size; 11 presented a power calculation which in all but one related to efficacy (the exception being the internet trial which related to correlation between alternative scales for assessing understanding of treatment consequences [20]); and 3 included non-statistical justifications (one stated the size to be adequate for a pilot study providing the chance to see if there were trends between active and placebo arms [53], the second didn't present a power calculation on the grounds that it was a pilot study but the size of 60 was based on safety data for other indications and was an achievable number [28], while the third was designed as a pilot, no significant differences were anticipated, and the size was chosen based on feasibility for a single-site study [10]). The median achieved sample size was 34 but there were three large trials: the first recruited 425 adolescents to test a sleep educational program in secondary schools [23]; the internet trial [20] recruited 998 people after sending out approximately 700,000 emails; and 3,318 people were recruited from 653,417 information packs mailed in a screening trial for lung cancer [32].
Figure 2 presents the percentage of papers in which methodological issues, efficacy, and safety/toxicity were explicitly stated as objectives, addressed with numerical results, discussed to an extent rated greater than minimal, or where conclusions were drawn. High percentages relating to efficacy can be seen for both drug and non-drug trials. The drug trials also addressed safety/toxicity issues, whereas the non-drug trials were more likely to additionally address methodological issues. In Table 3 the specific methodological issues are detailed. Recruitment and retention were frequently, though not always addressed to some extent: most papers with tabular/graphical presentation covered the issues in a CONSORT flowchart. Compliance/adherence to intervention was included in a few CONSORT flowcharts, but was more frequently addressed in text. Other aspects of intervention examined included cost and duration. The one paper that presented tabulated/graphical results relating to outcome assessment portrayed values elicited with category rating scales mapped onto a visual analogue scale format [20]. We rated the average costs per randomization presented in one paper [49] as detailed results relating to randomization procedures, the two papers with brief results on this topic commented on a failure in the randomization service [12] and gave numbers and reasons why potential participants missed being randomized [38]. Although our sample included 19 multi-centre trials only one presented numerical results evaluating procedures involved (ratings of cooperation between different centres [49]).
Percentage of papers in which feasibility, efficacy and safety/toxicity objectives, results, discussion or conclusions were presented (percentages for the Conclusions section based on the 32 papers including one).
The emphasis on efficacy demonstrated in Figure 2 is explored in Table 4. Statistically significant differences between groups were reported in 19 (38%) of papers, but we were unable to determine whether these related to pre-stated primary outcomes or were opportunistic reporting. Where significant differences between groups were not reported sometimes significant within group differences were. In 43 (86%) the Discussion section included a statement that the trial was too small: a similar statement was made in 24 (48%) of the Abstracts. Efficacy was addressed in the Discussion by all authors and in 26 (52%) it was the major focus. The take home message on efficacy from the Abstract section was rated to be that the intervention was beneficial in 24 (48%), and showed promise in 13 (26%). In 8 (16%, CI 7% to 28%) of the Abstracts there was no mention of efficacy.
As shown in Table 5, while most authors (39, 78%, CI 64% to 88%) did mention methodological issues in the Discussion it was often to a minimal extent. We rated the discussion to be substantial or the major focus of the section for 28 (56%, CI 41% to 70%), and covered to greater depth in the papers reporting non-drug compared to drug trials (P = 0.002). Most papers mentioned future trials but it was usually to an extent we rated as minimal: as with the coverage of feasibility issues more generally the depth of discussion concerning future trials was rated to be greater in the paper reporting non-drug trials (P = 0.002). Papers reporting non-drug trials were also more likely to include a methodological conclusion in the Conclusions section if there was one (P = 0.022), and in the Abstract (P = 0.031). The groups were similar with respect to whether the authors were actually conducting a subsequent trial with only 6 (12%, CI 5% to 23%) overall stating that one was underway or scheduled to start in the near future.
Discussion
For the main part the pilot and feasibility trials in our review did not primarily address methodological issues. Although lessons learnt about planning trials were discussed in the majority of papers it was often to an extent we rated as minimal, and in only 6 (12%) of papers was it stated that the authors were actually conducting a subsequent trial or about to start one. This frequency is not dissimilar to the 9% of pilot studies reviewed by Lancaster et al [2], subsequently found to have been followed by a larger study [4]. Even though authors themselves may not proceed to a larger RCT it is possible that others reading the paper will. Our impression was that many of the trials fell into the latter of Loscalzo's [9] two classes: namely those designated as pilots a posteriori possibly after failing to demonstrate the hoped for effects or because of inadequate sample size. In all but one of the 11 papers including a power calculation, sample size was determined to achieve power in testing efficacy. In other papers it was impossible to be sure what a priori objectives were from the published paper alone, but since methodological issues were discussed to greater depth in the non-drug trials, they are more likely to fall into Loscalzo's class of a priori pilot trials. Even amongst the non-drug trials there was generally an emphasis on efficacy.
We interpreted efficacy as the examination of change in an outcome variable not clearly related to safety/toxicity. The final group of scientific objectives for pilot studies listed by Thabane et al [5] includes obtaining estimates of the treatment effect and its variance. Thabane et al also discuss the distinction between pilot and proof-of-concept studies which they define as a clinical trial carried out to determine if a treatment (drug) is biologically active or inactive. Arnold et al [3] include the assessment of mechanisms, possibly using surrogate measures, as a legitimate objective of pilot trials, to establish proof-of-principal and potential efficacy. Many of the papers in our review may lie on the margins between pilot and proof-of-concept investigations and their emphasis on efficacy should perhaps be interpreted in this light, however none were described as proof-of- either concept or principal studies in their title [10 - 59], though one [42] was described as a phase II pilot study.
We specifically selected pilot/feasibility trials that incorporated random allocation. Many methodological issues do not need to be examined in the context of an RCT: for example larger numbers would be available from routinely collected data; it is generally easier to conduct a single group study; and greater depth of understanding of the acceptability of interventions is obtained from qualitative research. Some issues that cannot be satisfactorily investigated other than in the context of a randomized trial are the percentage consenting to randomization, retention in intervention and control groups, whether blinding can be maintained, and whether all components of the protocol work together. Given the burden of research governance concerning RCTs, it would seem sensible to evaluate specific aspects of a protocol using simpler studies wherever possible. In Table 6 we indicate with a tick methodological issues that require piloting in the context of an RCT, those marked with a cross could be assessed in other types of feasibility study.
Thabane et al [5] recommend that explicit criteria indicating that a subsequent trial is feasible should be stated: they describe the criteria set out in advance for proceeding from the pilot to the main Prophylaxis of Thromboembolism in Critical Care Trial (PROTECT) [62], also reviewed by Arnold et al [3]. In contrast, Gardener et al [63] describe a case study where unanticipated problems arising during a pilot lead to abandoning a subsequent RCT even though pre-stated objectives indicated the methodology to be feasible. They identify the availability of funding and the contemporary health service environment as issues likely to impact on the decision to proceed. We believe that studies evaluating the feasibility of trial procedures are essentially exploratory in nature. Researchers should examine carefully the success of procedures and react to unanticipated problems to get the best possible design for their next trial. Aspects of the design aren't decided in isolation, predicted recruitment under a set of eligibility criteria may be adequate if there is a change in outcome variable for example. If extensive changes are made it may be advisable to retest the feasibility of the protocol.
Others have searched for pilot/feasibility trials adopting different criteria leading to different populations of papers surveyed. Lancaster et al [2] could find no guidance on how to search MEDLINE for pilot/feasibility trials, and restricted their search to papers in six top ranking medical journals with the words 'pilot' or 'feasibility' in the title, abstract or keywords: of the 115 hits retrieved 25 (22%) were not suitable for a variety of reasons. Arain et al [4] repeated Lancaster et al's search procedure seven years later with a rate of unsuitable papers (30%) similar to ours of 33%. Arnold et al [3] initially searched MEDLINE for pilot trials in critical care medicine, but then canvassed known clinical investigators because of the poor indexing of pilot trials. They describe five pilot trials fulfilling the requirements of either addressing methodological issues relating to the feasibility of subsequent trials, or assessing mechanisms of intervention. Reviews based on published papers do not address internal pilots which continue into the main phase and are unlikely to be reported separately, though where a decision is taken not to continue the pilot phase could be written up. We chose to include only full papers in our study: different issues may arise amongst pilot or feasibility trials that are published as letters or brief reports. Sampling from all journals covered by MEDLINE and EMBASE and restricted to trials incorporating random allocation, the characteristics of the pilot/feasibility trials we found is not unexpected. The majority bear little resemblance to the recent definitions proposed for pilot/feasibility studies.
Conclusions
Our main findings are that RCTs described by their authors as pilots or addressing feasibility most commonly focus on efficacy, in just over a half (56%) were issues that might inform the planning of a subsequent trial addressed in reasonable depth. In addition to efficacy pilot drug trials also addressed safety, while pilot non-drug trials were more likely to additionally address methodological issues. While the median sample size was quite small at 34, there were three trials recruiting over 100 participants, demonstrating that sample size very much depends on circumstances even in the context of pilot/feasibility trials.
References
- 1.
van Teijlingen ER, Hundley V: The Importance of Pilot Studies. Social Research Update. 2001, 35: [http://sru.soc.surrey.ac.uk/SRU35.html]
- 2.
Lancaster GA, Dodd S, Williamson PR: Design and analysis of pilot studies: recommendations for good practice. Journal of Evaluation in Clinical Practice. 2004, 10: 307-312. 10.1111/j..2002.384.doc.x.
- 3.
Arnold DM, Burns KE, Adhikari NKH, Kho ME, Meade MO, Cook DJ: The design and interpretation of pilot trials in clinical research in critical care. Critical Care Medicine. 2009, 37: S69-S74. 10.1097/CCM.0b013e3181920e33.
- 4.
Arain M, Campbell MJ, Cooper CL, Lancaster GA: What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Medical Research Methodology. 2010, 10: 67-10.1186/1471-2288-10-67.
- 5.
Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, Robson R, Thabane M, Giangregorio L, Goldsmituh CH: A tutorial on pilot studies: the what, why and how. BMC Medical Research Methodology. 2010, 10: 1-10.1186/1471-2288-10-1.
- 6.
NIHR Health Technology Assessment Programme (NETSCC): Feasibility and Pilot studies. 2011, Last accessed June 27, [http://www.netscc.ac.uk/glossary/index.asp#glos7]
- 7.
Loscalzo J: Pilot trials in clinical research: Of what value are they?. Circulation. 2009, 119: 1694-1696. 10.1161/CIRCULATIONAHA.109.861625.
- 8.
Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P: Framework for design and evaluation of complex interventions to improve health. BMJ. 2000, 321: 694-6. 10.1136/bmj.321.7262.694.
- 9.
Craig P, Dieppe P, Macintyre , Mitchie , Nazareth I, Petticrew M: Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008, 337: 979-983. 10.1136/bmj.a979.
- 10.
Aisen PS Schmeidler J, Pasinetti GM: Randomized pilot study of nimesulide treatment in Alzheimer's disease. Neurology. 2002, 58: 1050-4.
- 11.
Andreatini R, Sartori VA, Seabra MLV, Leite JR: Effect of valepotriates (valerian extract) in generalized anxiety disorder: a randomized placebo-controlled pilot study. Phytotherapy Research. 2002, 16: 650-4. 10.1002/ptr.1027.
- 12.
Barrett JA, Watkins C, Plant R, Dickinson H, Clayton L, Sharma AK, Reston A, Gratton J, Fall S, Flynn A, Smith T, Leathley M, Smith S, Barer DH: The COSTAR wheelchair study: a two-centre pilot study of self-propulsion in a wheelchair in early stroke rehabilitation. Clinical Rehabilitation. 2001, 15: 32-41. 10.1191/026921501672264719.
- 13.
Bean J, Herman S, Kiely DK, Callahan D, Mizer K, Frontera WR, Fielding RA: Weighted stair climbing in mobility-limited older people: a pilot study. Journal of the American Geriatrics Society. 2002, 50: 663-70. 10.1046/j.1532-5415.2002.50160.x.
- 14.
Beatty G, Hunt P, Smith A, Hoh R, Huang W, Martin J, Deeks SG: A randomized pilot study comparing combination therapy plus enfuvirtide versus a treatment interruption followed by combination therapy plus enfuvirtide. Antiviral Therapy. 2006, 11: 315-9.
- 15.
Bernstein DI, Spruance SL, Arora SS, Schroeder JL, Meng T-C: Evaluation of imiquimod 5% cream to modify the natural history of herpes labialis: a pilot study. Clinical Infectious Diseases. 2005, 41: 808-14. 10.1086/432802.
- 16.
Beysen C, Murphy EJ, Nagaraja H, Decaris M, Riiff T, Fong A, Hellerstein MK, Boyle PJ: A pilot study of the effects of pioglitazone and rosiglitazone on de novo lipogenesis in type 2 diabetes. Journal of Lipid Research. 2008, 49: 2657-63. 10.1194/jlr.M800165-JLR200.
- 17.
Blake H, Batson M: Exercise intervention in brain injury: a pilot randomized study of Tai Qigong. Clinical Rehabilitation. 2009, 23: 589-98. 10.1177/0269215508101736.
- 18.
Buckley T, Schalzberg AF: The acute and post-discontinuation effects of a glucocorticoid receptor (GR) antagonist probe on sleep and the HPA axis in chronic insomnia: a pilot study. Journal of Clinical Sleep Medicine. 2008, 4: 235-241.
- 19.
Callaghan MJ, Oldham JA, Winstanley J: A comparison of two types of electrical stimulation of the quadriceps in the treatment of patellofemoral pain syndrome. a pilot study. Clinical Rehabilitation. 2001, 15: 637-46. 10.1191/0269215501cr457oa.
- 20.
Carling C, Kristoffersen DT, Herrin J, Treweek S, Oxman AD, Schünemann H, Aki EA, Montori V: How should the impact of different presentations of treatment effects on patient choice be evaluated? A pilot randomized trial. PLoS ONE. 2008, 11: e3693-
- 21.
Chandra P, Binongo JNG, Ziegler TR, Schlanger LE, Wang W, Someren JT, Tangpricha V: Cholecalciferol (vitamin D3) therapy and vitamin D insufficiency in patients with chronic kidney disease: a randomized controlled pilot study. Endocrine Practice. 2008, 14: 10-7.
- 22.
Christensen ER, Stegger M, Jensen-Fangel S, Laursen AL, Ostergaard L: Mitochondrial DNA levels in fat and blood cells from patients with lipodystrophy or peripheral neuropathy and the effect of 90 days of high-dose coenzyme Q treatment: a randomized, double-blind, placebo-controlled pilot study. Clinical Infectious Diseases. 2004, 39: 1371-9. 10.1086/424881.
- 23.
Cortesi F, Giannotti F, Sebastiani T, Bruni O, Ottaviano S: Knowledge of sleep in Italian high school students: pilot-test of a school-based sleep educational program. Journal of Adolescent Health. 2004, 34: 344-351.
- 24.
Crogan NL, Evans BC, Bendel R: Storytelling intervention for patients with cancer: part 2-pilot testing. Oncology Nursing Forum. 2008, 35: 265-72. 10.1188/08.ONF.265-272.
- 25.
Dass S, Bowman SJ, Vital EM, Ikeda K, Pease CT, Hamburger J, Richards A, Rauz S, Emery P: Reduction of fatigue in Sjogren syndrome with rituximab: results of a randomised, double-blind, placebo-controlled pilot study. Annals of the Rheumatic Diseases. 2008, 67: 1541-4.
- 26.
de Geest S, Schafer-Keller P, Denhaerynck K, Thannberger N, Kofer S, Bock A, Surber C, Steiger J: Supporting medication adherence in renal transplantation (SMART): a pilot RCT to improve adherence to immunosuppressive regimens. Clinical Transplantation. 2006, 20: 359-68. 10.1111/j.1399-0012.2006.00493.x.
- 27.
de Lusignan S, Wells S, Johnson P, Meredith K, Leatham E: Compliance and effectiveness of 1 year's home telemonitoring. The report of a pilot study of patients with chronic heart failure. European Journal of Heart Failure. 2001, 3: 723-30. 10.1016/S1388-9842(01)00190-8.
- 28.
diZerega GS, Verco SJS, Young P, Kettel M, Kobak W, Martin D, Sanfilippo J, Peers EM, Scrimgeour A, Brown CB: A randomized, controlled pilot study of the safety and efficacy of 4% icodextrin solution in the reduction of adhesions following laparoscopic gynaecological surgery. Human Reproduction. 2002, 17: 1031-8. 10.1093/humrep/17.4.1031.
- 29.
Farbu E, Rekand T, Vik-Mo E, Lygren , Gilhus NE, Aarli JA: Post-polio syndrome patients treated with intravenous immunoglobin: a double-blinded randomized controlled pilot study. European Journal of Neurology. 2007, 14: 60-65. 10.1111/j.1468-1331.2006.01552.x.
- 30.
Fuchs S, Dib N, Cohen BM, Okubagzi P, Diethrich EB, Campbell A, Macko J, Kessler PD, Rasmussen HS, Epstein SE, Kornowski R: A randomized, double-blind, placebo-controlled, multicenter, pilot study of the safety and feasibility of catheter-based intramyocardial injection of AdVEGF121 in patients with refractory advanced coronary artery disease. Catheterization and Cardiovascular Interventions. 2006, 68: 372-8. 10.1002/ccd.20859.
- 31.
Fürst A, Stuttner A, Agha A, Behan A, Jauch K-W: Colonic J-pouch vs. coloplasty following resection of distal rectal cancer Early results of a prospective, randomized, pilot study. Diseases or the Colon & Rectum. 2003, 46: 1161-1166. 10.1007/s10350-004-6707-x.
- 32.
Gohagan J, Marcus P, Fagerstrom R, Pinsky P, Kramer B, Prorok P: Baseline findings of a randomized feasibility trial of lung cancer screening with spiral CT scan vs chest radiograph: the Lung Screening Study of the National Cancer Institute. Chest. 2004, 126: 114-21. 10.1378/chest.126.1.114.
- 33.
Grenander A, Bredbacka S, Rydvall A, Aroch R, Edner G, Koskinen LOD, Olivecrona M: Antithrombin treatment in patients with traumatic brain injury: a pilot study. Journal of Neurosurgical Anesthesiology. 2001, 13: 49-56. 10.1097/00008506-200101000-00010.
- 34.
Hayton MJ, Santing AJA, Hughes PJ, Frostock SP, Trail IA, Stanley JK: Botulinum toxininjection in the treatment of tennis elbow. A double-blind, randomized, controlled, pilot study. Journal of Bone and Joint Surgery. 2005, 87-A: 502-507.
- 35.
Jain S, Dawson J, Quinn NP, Playford ED: Occupational therapy in multiple system atrophy: a pilot randomized controlled trial. Movement Disorders. 2004, 19: 1360-4. 10.1002/mds.20211.
- 36.
Janelli LM, Kanski GW, Wu YW: The influence of individualized music on patients in physical restraints: a pilot study. The Journal of the New York State Nurses' Association. 2005, 35: 22-7.
- 37.
Karleen Su T-I, Khanna D, Furst DE, Danovitch G, Burger C, Maranian P, Clements PJ: Rapamycin versus methrotrexate in early diffuse systematic sclerosis: results from a randomized, single-blind pilot study. Arthritis and Rheumatism. 2009, 60: 3821-30. 10.1002/art.24986.
- 38.
Khodabux CM, von Lindern JS, van Hilten JA, Scherjon S, Walther FJ, Brand A: A clinical study on the feasibility of autologous cord blood transfusion for anemia of prematurity. Transfusion. 2008, 48: 1634-43. 10.1111/j.1537-2995.2008.01747.x.
- 39.
Klais CM, Eandi CM, Ober MD, Sorenson JA, Sadeghi SN, Freund KB, Spaide RF, Slakter JS, Yannuzzi LA: Anecortave acetate treatment for retinal angiomatous proliferation: a pilot study. Retina. 2006, 26: 773-9. 10.1097/01.iae.0000244261.52901.67.
- 40.
Klarin B, Molin G, Jeppsson B, Larsson A: Use of probiotic Lactobacillus plantarum 299 to reduce pathogenic bacteria in the oropharynx of intubated patients: a randomized controlled open pilot study. Critical Care. 2008, 12: R136-10.1186/cc7109.
- 41.
Lannin NA, Clemson L, McCluskey A, Lin CW, Cameron ID, Barras S: Feasibility and results of a randomised pilot-study of pre-discharge occupational therapy home visits. BMC Health Sservices Research. 2007, 7: 42-10.1186/1472-6963-7-42.
- 42.
Lim DA, Tarapore P, Chang E, Burt M, Chakalian L, Barbaro N, Chang S, Lamborn KR, McDermott MW: Safety and feasibility of switching from phenytoin to levetiracetam monotherapy for glioma-related seizure control following craniotomy: a randomized phase II pilot study. Journal of Neurooncology. 2009, 93: 349-54. 10.1007/s11060-008-9781-4.
- 43.
Luiten RM, Kueter EW, Mooi W, Gallee MP, Rankin EM, Gerritsen WR, Clift SM, Nooijen WJ, Weder P, van de Kasteele WF, Sein J, van den Berk PC, Nieweg OE, Berns AM, Spits H, de Gast GC: Immunogenicity, including vitiligo, and feasibility of vaccination with autologous GM-CSF-transduced tumor cells in metastatic melanoma patients. Journal of Clinical Oncology. 2005, 23: 8978-91. 10.1200/JCO.2005.01.6816.
- 44.
Madden JAJ, Plummer SF, Tang J, Garaiova I, Plummer NT, Herbison M, Hunter JO, Shimada T, Cheng L, Shirakawa T: Effect of probiotics on preventing disruption of the intestinal microflora following antibiotic therapy: a double-blind, placebo-controlled pilot study. International Immunopharmacology. 2005, 5: 1091-7. 10.1016/j.intimp.2005.02.006.
- 45.
McCrossan B, Morgan G, Grant B, Sands A, Craig B, Casey F: Assisting the transition from hospital to home for children with major congenital heart disease by telemedicine: a feasibility study and initial results. Medical informatics and the Internet in Medicine. 2007, 32: 297-304. 10.1080/14639230701791611.
- 46.
Morriss R, Dowrick C, Salmon P, Peters S, Rogers A, Dunn G, Lewis B, Charles-Jones H, Hogg J, Clifforda R, Iredale W, Towey M, Gask L: Turning theory into practice: rationale, feasibility and external validity of an exploratory randomized controlled trial of training family practitioners in reattribution to manage patients with medically unexplained symptoms (the MUST). General Hospital Psychiatry. 2006, 28: 343-51. 10.1016/j.genhosppsych.2006.03.012.
- 47.
Mustian KM, Katula JA, Zhao H: A pilot study to assess the influence of Tai Chi Chuan on functional capacity among breast cancer survivors. Journal of Supportive Oncology. 2006, 4: 139-145.
- 48.
Rasmussen NA, Schroder P, Olsen LR, Brodsgaard M, Unden M, Bech P: Modafinil augmentation in depressed patients with partial response to antidepressants: a pilot study on self-reported symptoms covered by the Major Depression Inventory (MDI) and the Symptom Checklist (SCL-92). Nordic Journal of Psychiatry. 2005, 59: 173-8. 10.1080/08039480510023025.
- 49.
Rowe DE, Feise RJ, Crowther ER, Grod JP, Menke JM, Goldsmith CH, Stoline MR, Souza TA, Kambach B: Chiropractice manipulation in adolescent idiopathic scoliosis: a pilot study. BMC Chiropractic & Osteopathy. 2006, 14: 15-10.1186/1746-1340-14-15.
- 50.
Sheehan JL, Winzeler-Mercay U, Mudie MH: A randomized controlled pilot study to obtain the best estimate of the size of the effect of a thermoplastic resting splint on spasticity in the stroke-affected wrist and fingers. Clinical Rehabilitation. 2006, 20: 1032-7. 10.1177/0269215506071267.
- 51.
Shoptaw S, Watson DW, Reiber C, Rawson RA, Montgomery MA, Majewska MD, Ling W: Randomized controlled pilot trial of cabergoline, hydergine and levodopa/carbidopa: Los Angeles Cocaine Rapid Efficacy Screening Trial (CREST). Addiction. 2005, 100 (SUPPL 1): 78-90.
- 52.
Simpson TL, Saxon AJ, Meredith CW, Malte CA, McBride B, Ferguson LC, Gross CA, Hart KL, Raskind M: A pilot trial of the alpha-1 adrenergic antagonist, prazosin, for alcohol dependence. Alcoholism: Clinical and Experimental Research. 2009, 33: 255-63. 10.1111/j.1530-0277.2008.00807.x.
- 53.
Stiasny-Kolster K, Kohnen R, Schollmayer E, Moller JC, Oertel WH, the Rotigotine SP 666 Study Group: Patch application of the dopamine agonist rotigotine to patients with moderate to advanced stages of restless legs syndrome: a double-blind, placebo-controlled pilot study. Movement Disorders. 2004, 19: 1432-8. 10.1002/mds.20251.
- 54.
Stoel MG, Marques KMJ, de Cook CC, Bronzwaer JGF, von Birgelen C, Zijlstra F: High dose adenosine for suboptimal myocardial reperfusion after primary PCI: a randomized placebo-controlled pilot study. Catheterization and Cardiovascular Interventions. 2008, 71: 283-289.
- 55.
Todd MK, Reis-Bergan MJ, Sidman CL, Flohr JA, Jameson-Walker K, Spicer-Bartolau T, Wildeman K: Effect of a family-based intervention on electronic media use and body composition among boys aged 8--11 years: a pilot study. Journal of Child Health Care. 2008, 12: 344-58. 10.1177/1367493508097404.
- 56.
Tsai H-J, Hung H-C, Yang J-L, Huang C-S, Tsauo J-Y: Could kinesio tape replace the bandage in decongestive lymphatic therapy for breast-cancer-related lymphedema? A pilot study. Supportive Care in Cancer. 2009, 17: 1353-60. 10.1007/s00520-009-0592-8.
- 57.
van Royen N, Schirmer SH, Atasever B, Behrens CY, Ubbink D, Buschmann EE, Voskuil M, Bot P, Hoefer I, Schlingemann RO, Biemond BJ, Tijssen JG, Bode C, Schaper W, Oskam J, Legemate DA, Piek JJ, Buschmann I: START Trial: a pilot study on STimulation of ARTeriogenesis using subcutaneous application of granulocyte-macrophage colony-stimulating factor as a new treatment for peripheral vascular disease. Circulation. 2005, 112: 1040-6. 10.1161/CIRCULATIONAHA.104.529552.
- 58.
Werk M, Langner S, Reinkensmeier B, Boettcher H-F, Tepe G, Dietz U, Hosten N, Hamm B, Speck U, Ricke J: Inhibition of restenosis in femoropopliteal arteries. Paclitaxel-coated versus uncoated balloon: Femoral paclitaxel randomized pilot trial. Circulation. 2008, 118: 1358-65. 10.1161/CIRCULATIONAHA.107.735985.
- 59.
Yadav V, Marracci G, Lovera J, Woodward W, Bogardus K, Marquardt W, Shinto L, Morris C, Bourdette DN: Lipoic acid in multiple sclerosis: a pilot study. Multiple Sclerosis. 2005, 11: 159-65. 10.1191/1352458505ms1143oa.
- 60.
Schulz KF, Altman DG, Moher D: CONSORT 2010 Statement; updated guidelines for reporting parallel group randomised trials. BMJ. 2010, 340: 698-702.
- 61.
Cytel Software Corporation: StatXact: Version 8. 2007, Cambridge, MA 02139
- 62.
Cook DJ, Rocker G, Meade M, Guyatt G, Geerts W, Anderson D, Skrobik Y, Hebert P, Albert M, Cooper J, Bates S, Caco C, Finfer S, Fowler R, Freitag A, Granton J, Jones G, Langevin S, Mehta S, Pagliarello G, Poirier G, Rabbat C, Schiff D, Griffith L, Crowther M, the Canadian Critical Care Trials Group: Prophylaxis of Thromboembolism in Critical Care (PROTECT) Trial: a pilot study. Journal of Critical Care. 2005, 20: 364-372. 10.1016/j.jcrc.2005.09.010.
- 63.
Gardner G, Gardner A, MacLellan L, Osborne S: Reconceptualising the objectives of a pilot study for clinical research. International Journal of Nursing Studies. 2003, 40: 719-724. 10.1016/S0020-7489(03)00011-7.
Pre-publication history
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/11/117/prepub
Author information
Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors' contributions
MS carried out the literature search, created the review form and reviewed the papers under the supervision of RMP and MW. RMP and MW conceived the study, supervised its conduct and reviewed some of the papers. RMP drafted an initial version of the manuscript: all authors contributed to and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
About this article
Cite this article
Shanyinde, M., Pickering, R.M. & Weatherall, M. Questions asked and answered in pilot and feasibility randomized controlled trials. BMC Med Res Methodol 11, 117 (2011). https://doi.org/10.1186/1471-2288-11-117
Received:
Accepted:
Published:
Keywords
- Methodological Issue
- Pilot Trial
- Drug Trial
- Subsequent Trial
- Feasibility Trial
