- Open Access
A novel simulation-based approach to training for recruitment of older adults to clinical trials
BMC Medical Research Methodology volume 22, Article number: 180 (2022)
The need to engage adults, age 65 and older, in clinical trials of conditions typical in older populations, (e.g. hypertension, diabetes mellitus, Alzheimer’s disease and related dementia) is exponentially increasing. Older adults have been markedly underrepresented in clinical trials, often exacerbated by exclusionary study criteria as well as functional dependencies that preclude participation. Such dependencies may further exacerbate communication challenges. Consequently, the evidence of what works in subject recruitment is less generalizable to older populations, even more so for those from racial and ethnic minority and low-income communities.
To support capacity of research staff, we developed a virtual, three station simulation (Group Objective Structured Clinical Experience—GOSCE) to teach research staff communication skills. This 2-h course included a discussion of challenges in recruiting older adults; skills practice with Standardized Participants (SPs) and faculty observer who provided immediate feedback; and debrief to highlight best practices. Each learner had opportunities for active learning and observational learning. Learners completed a retrospective pre-post survey about the experience. SP completed an 11-item communication checklist evaluating the learner on a series of established behaviorally anchored communication skills (29).
In the research staff survey, 92% reported the overall activity taught them something new; 98% reported it provided valuable feedback; 100% said they would like to participate again. In the SP evaluation there was significant variation: the percent well-done of items by case ranged from 25–85%.
Results from this pilot suggest that GOSCEs are a (1) acceptable; (2) low cost; and (3) differentiating mechanism for training and assessing research staff in communication skills and structural competency necessary for participant research recruitment.
Despite an increasing need to recruit older clinical trial participants, adults over 65, in particular, are markedly underrepresented. Research participation has steadily declined over the past 30 years [1,2,3]. Racial or ethnic minorities are particularly underrepresented in clinical research [4,5,6]. Consequently, the evidence of what treatment and intervention strategies work in participant recruitment is less generalizable to older populations, and even less so for those from racial and ethnic minority and low-income communities. This is particularly troubling for clinical trials that study conditions such as hypertension, diabetes mellitus, Alzheimer’s disease and related dementia typical in older populations and more prevalent among minority groups [7, 8].
Establishing trust and rapport are fundamental to effective recruitment of older adults into research ; Atwood found staff rapport was one of the top four motivations for participation among older adults . Dimensions of research staff, such as communication, flexibility, respect, and appreciation are reported to be among the leading reasons why older adults ages 65 and older remained in studies [11, 12]. Yet structural competency – i.e. the capacity of health professionals to understand the psychosocial and health context of older adults – is limited among research teams . One barrier to recruitment is inattention to specific characteristics that older age participants face and that differ from younger age populations. Such characteristics necessitate different recruitment strategies [4, 13]. These include allowing verbal responses for those with manual dexterity challenges, allotting extra time for those with slower cognitive processing speed, and employing written materials for those with hearing impairment, as some salient examples . Research staff are often tasked with recruitment and need communication skills to respond to these challenges. Nishimura et al. suggest that training to improve communication skills might be important in enhancing participant consent .
While studies have suggested that “education,” defined as materials or classroom learning, does not increase the efficacy of recruiters or research assistants in recruiting individuals to trials, there is little information on the feasibility or potential efficacy of simulation-based educational experiences in training these recruiters . In a 2013, meta-analysis of 45 interventions to improve recruitment to clinical trials, just two considered interventions aimed at the recruiters themselves . Of those two, one trial increased recruitment, one trial had no impact, but both educational interventions focused only on didactic learning. Meta-analysis authors concluded that there was a crucial knowledge gap in recruitment interventions aimed at recruiters . Simulation and performance based training with trained actors playing standardized patients are used in medical education to teach communication skills. The Group Objective Structured Clinical Experience (GOSCE) is a simulation-based experience in which learners move in groups to practice and then reflect on their communication skills. GOSCEs address that critical gap in recruitment interventions aimed at recruiters by aiming an intervention at recruiters. In an effort to address low recruitment of older adults, we sought to assess the acceptability of implementing a simulated educational experience, one of the most widely used methods of assessing and developing aspects of clinical competency in healthcare education, for research assistants .
We developed a quality improvement project that consisited of a three station GOSCE where research staff practiced a recruitment protocol for a hypothetical clinical trial (in which older adults would receive weekly visits with a puppy) using simulated older adult participants. Table 1 outlines the GOSCE structure. Through the GOSCE we sought to enhance recruiters’ communication skills and capacity to engage, build trust and ultimately enroll older participants into research. We chose this simulation methodology based on evidence that practicing these skills and receiving immediate feedback is a high yield strategy for strengthening communication skills in physician and non-physician healthcare workers [17,18,19]. Our objective was to assess the acceptability of GOSCEs to train research staff in best practices for recruiting and consenting older adults to clinical trials.
Materials and methods
GOSCEs are a more cost-and resource-effective [20, 21] variation on the traditional objective structured clinical experience (OSCE), which is a standard component in undergraduate and graduate medical education curricula and is used in many healthcare fields, including midwifery , physical and occupational therapy , nursing , and research [22, 25]. In a GOSCE learners move in groups through stations. In each station, one group member has the opportunity to lead the interaction with the standardized patient (a professional actor trained to play a patient), while others have a role observing [26, 27]. After the encounter, there is an opportunity for standardized patient (SP) perspective, peer debrief and feedback from a faculty observer which strengthens the GOSCE’s value as a formative assessment tool [20, 21, 28, 29]. This model, of concrete experience (with SP) followed by reflective observation (with the group and faculty member) is modeled after David Kolb’s experiential learning theory .
Two experienced medical educators with expertise in performance-based training and assessment designed the quality improvement, GOSCE experience with input from an IRB expert, a geriatrician, and a PhD researcher specializing in recruitment and retention in clinical trials, in consultation with leaders of community organizations for older adults. The scenarios for the simulated recruitment of older adults were 1) a Black woman with hearing impairment 2) a white woman and her family member both present with differing views on research participation, and 3) a Black man with concerns about participating given a history of racism in medical research. Cases were developed based on a literature review of facilitators and barriers to clinical trial recruitment and a focus group of 5 research staff who regularly recruit older adults to clinical trials. 4 SPs (one for each case, with two SPs for one case) were recruited from the pool of SPs employed by the medical school; they received 4 h of training (2 on the case portrayal, 2 on the checklist completion) by a physician with experience in performance-based assessment. Each item on the checklist is scored on a 3-point scale (not done, partly done and well done, each with a behavioral anchor describing the point). For example, “did not discuss risks,” “discussed risks BUT did not check for understanding or questions,” discussed risks AND checked for understanding or questions.’ The checklist is the standard model used in medical education at our site over the past 15 years and has good reliability and validity . Competencies we aimed to assess include: building trust and rapport with participant; assessing understanding and capacity to consent; and presenting information. Cases were shared with experts in the clinical trial recruitment field and modified accordingly until expert consensus on the cases was reached.
In this GOSCE, the convenience sample included 45 research staff at a large, urban hospital, both staff who routinely recruit older adults to participate in clinical trials and staff who are involved in recruitment training and strategy. The GOSCE was conducted five times, each with 9 participants. The total cost of each GOSCE was 800 US dollars. GOSCE participants had been recruiters for less than two years. Using a virtual conferencing platform, this 2-h GOSCE included three sections: 1) a 30 min orientation to GOSCE with discussion on challenges and facilitators to obtaining consent and a model for effective recruitment; 2) three 20 min GOSCE stations with Standardized Participants (SPs) with 10 min for the interview and 10 min for immediate feedback; and 3) a 25 min group debrief to review experiences, highlight best practices and help integrate new approaches into behavioral repertoires. After the initial orientation, Research staff rotated in groups of three through three stations, each with an SP whom they needed to recruit to the “trial. An observing faculty member provided immediate feedback after this simulated recruitment effort and led a debrief with all three members of the group. During this debrief, the SP completed an 11-item communication checklist evaluating the learner on a series of behaviorally anchored communication skills; after finishing the checklist, the SP would join the debrief and share additional feedback based on the checklist. Each learner had opportunities for active learning and observational learning. After these encounters, the group came back together with all the faculty observers for a group discussion and debrief. A handout on best practices, developed by the team of medical educators and researcher with experience in clinical recruitment, was shared for participants to use in their future work.
The program was evaluated by research staff through a 36-question survey including retrospective pre-post items (i.e. learners completed the survey after the GOSCE reflecting on skills before and after GOSCE) that used 3 and 4-point Likert scale questions (i.e. not at all skilled to very skilled, low educational value to high educational value). Free text questions about experience with the cases and lessons learned from the training were also included. Questions addressed: 1) self-assessed change in skill after the workshop; 2) new discoveries in recruitment; and 3) overall educational value. The survey was adapted from a standard educational evaluation survey used in simulation training at our institution to elicit acceptability, relevance and change in knowledge and attitudes. Effectiveness of program was evaluated through Kirkpatrick’s training evaluation model Level 1 (Reaction) and Level 2 (Learning) .
This survey was distributed through an anonymous link post-GOSCE. None of the faculty members involved in the cases and feedback session are the supervisors of any of the participants of the GOSCE. Descriptive statistics and a nonparametric sign test for median differences on changes in self-reported skills were computed for survey items. Free text responses were assessed using a qualitative thematic analysis.
The curriculum was delivered as planned, but given the advent of the COVID-19 pandemic, the GOSCE was switched to a remote (zoom) platform in order to be able to conduct safely. It was conducted three times: July 2020, October 2020, and April 2021. Ninety-one percent of research staff (41/45) completed the 36-item survey. Research staff recruit at a large, urban hospital with recruitment experience between 0 months and 2 years and represent a broad disciplines and range of research projects. Of these, 93% (38/41 reported the overall activity taught them something new; 98% (40/41) reported it provided valuable feedback; 100% (41/41) said they would like to participate again. 56% (23/41) percent reported perceived improvement in their recruitment skills as a result of the workshop.
As outlined in Table 2, research staff reported that these cases were similar to real life (range: 56–73% had encountered similar situation to the case they participated in) and they found all cases had high or moderate educational value in strengthening their skills. Representative quotes from the survey regarding the GOSCE’s educational value are also included in Table 2; these were identified through a brief thematic analysis conducted by two medical education researchers. The case learners were least familiar with was also the case they rated as having the highest educational value. Indeed, just 56% of research staff had experiences of the third case, in which the SP is hesitant about participating in research given the history of racism in medicine and his brother’s experience of medical neglect, but 82% of research staff reported it was of high educational value.
Seven research staff rated their pre-GOSCE skills as “very skilled” by 7 research staff; 23 rated themselves as “somewhat skilled”; 8 as “not very skilled”; and 3 as “not at all skilled”. Post-GOSCE, 20 rated themselves as “very skilled” and 21 rated skills as “somewhat skilled.” Self-rated skills improved for 23 of 41 research staff, while 18 research staff pre- and post-ratings were unchanged. An exact sign test was used to compare pre and post ratings. Scores post-GOSCE had a significantly different median increase compared to pre-GOSCE, p = 0.0001.
In the survey, participants listed their key learning points that aligned with the aims of the GOSCE. Table 3 outlines themes that emerged from these primary learning points. These were identified through an open-coded thematic analysis conducted by two medical education researchers. Themes included: learning about other, learning about self, specific behaviors to change, and learning by doing.
Due to internet challenges at the first GOSCE limiting completion of checklists, 39 of 45 communication checklists of participant performance during the GOSCE station (9 participants at each of 5 sessions) were completed by the SPs. Research staff performance on the 11-item communication checklist is reported in Table 4. These behaviorally anchored communication checklist items (Table 4) were developed by this team of medical educators and have demonstrated reliability . There was a range of scores: the % well-done of items by case ranged from 25–85%. For example, SPs reported that 85% of research staff “did not interrupt and allowed time to express thoughts fully” 25% checked your understanding through specific questioning and/or asking you to repeat back information.
The implementation of this GOSCE provides a novel methodology for increasing effective recruitment of older adults – but broadly all people – to clinical trials by aiming to increase recruitment through training recruiters (11). Results from this pilot suggest that GOSCEs are a (1) acceptable; (2) low cost; and (3) differentiating mechanism for training research staff in communication skills and structural competency necessary for participant research recruitment and identifying those that may need additional practice. This educational methodology is familiar to medical education, but novel for training those who recruit participants for clinical trials and addresses an important gap around recruitment training in the literature  (29). This data suggests that GOSCEs are acceptable trainings for research assistants and can be implemented in other settings to provide dynamic communication skills training.
Based on the retrospective pre-post, GOSCEs are feasible for training research staff in simulated clinical scenarios with which research staff reported they had experience; indeed, learners found this to be an educationally valuable experience (100% reported it was of high or moderate educational value, Table 2) and one that reflected situations they experience in practice (56–73% had encountered similar situation). This retrospective survey is standard practice in medical education and is a useful alternative to a pre-test where learners often over or under-estimate their abilities in (i.e. they don’t know what they don’t know), but pre-post tests are limited in that it may be influenced by social desirability and recall bias  (30). We selected three case scenarios based on interviews, literature review and relevance for these research staff, but this methodology is highly adaptable to a wide range of sensory, cultural, and individual challenges. GOSCEs are highly adaptable to the specific recruitment needs of a clinical trial and this work suggests they are useful for educating research staff.
GOSCEs are less expensive and time-intensive than some may expect. Simulation-based education is quite feasible, especially given that the vast majority of academic medical centers have simulation centers with whom one can partner to conduct this work. In addition, if SPs are not available, role-play can serve as a substitute for SPs. The same behaviorally anchored checklist to inform feedback to the learner can be used. For reference, the overall cost of training and employing 4 SPs for this GOSCE program (4 h training/actor + 2 h of GOSCE case/actor × 5 iterations) for the 45 research staff was approximately $2200.00. For comparison, conducting individual OSCEs for 45 research staff would cost at least $3400.00. These data suggest this innovation is an acceptable, low cost strategy for training research staff.
SP assessment of research staff identified variation in communication skills and suggested areas for further reinforcement to improve recruitment skills. The range of percent-well-done in SP evaluations indicates that GOSCEs can distinguish between higher and lower performing learners and provide behaviorally specific areas for feedback. In particular, the range of % well-done of items by case ranged from 25–85%. The lowest percent well done (25%) scores were the case where the SP had hearing loss, suggesting higher need for remediation skills that around accounts for auditory impairment of potential research participants. This variation in skills is notable given communication skills play a significant role in recruitment: a 2013 study that surveyed researchers and community members on the same questions about their practices and preferences during the informed consent process found that the ability to engage in one-on-one discussion about the study was most important for researchers and second most important (after take home materials) for community members, suggesting that identifying and strengthening skills around communication is central to effectively training those who recruit and consent participants to clinical trials .
While we are limited by lacking data on the impact this GOSCE had on participants’ overall recruitment success, we believe this interactive approach is likely to be more fruitful than the traditional classroom learning – a strategy that has not been found to be effective in increasing recruiters’ ability to recruit . We also limited in generalizability due to the sample and setting, potential limitations of the methods used to evaluate the GOSCE.
This pilot suggests that GOSCEs are a (1) acceptable; (2) low cost; and (3) differentiating mechanism for training research staff in communication skills and structural competency necessary for participant research recruitment. In addition, when surveyed, participants overwhelmingly felt it taught them new information and provided valuable feedback. Given the relatively low cost of implementing this program, we believe it has potential for researchers and research institutions looking to increase the participation of older adults in their research and clinical trials. Our next steps are to gather data on the impact of GOSCE training on research staff behavior during recruitment and consenting, and on research staff success in recruitment and research participants.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol. 2007;17(9):643–53.
Sedrak MS, et al. Older adult participation in cancer clinical trials: A systematic review of barriers and interventions. CA Cancer J Clin. 2021;71(1):78–92.
Shenoy P, Harugeri A. Elderly patients’ participation in clinical trials. Perspect Clin Res. 2015;6(4):184–9.
Mody L, et al. Recruitment and retention of older adults in aging research. J Am Geriatr Soc. 2008;56(12):2340–8.
Flores LE, et al. Assessment of the inclusion of racial/ethnic minority, female, and older individuals in vaccine clinical trials. JAMA Netw Open. 2021;4(2):e2037640–e2037640.
Amorrortu RP, et al. Recruitment of racial and ethnic minorities to clinical trials conducted within specialty clinics: an intervention mapping approach. Trials. 2018;19(1):115.
Mangoni AA, Jansen PA, Jackson SH. Under-representation of older adults in pharmacokinetic and pharmacodynamic studies: a solvable problem? Expert Rev Clin Pharmacol. 2013;6(1):35–9.
Michelet M, Lund A, Sveen U. Strategies to recruit and retain older adults in intervention studies: a quantitative comparative study. Arch Gerontol Geriatr. 2014;59(1):25–31.
Weil J, Mendoza AN, McGavin E. Recruiting older adults as participants in applied social research: applying and evaluating approaches from clinical studies. Educ Gerontol. 2017;43(12):662–73.
Atwood JR, et al. Reasons related to adherence in community-based field studies. Patient Educ Couns. 1992;19(3):251–9.
Forsat ND, et al. Recruitment and retention of older people in clinical research: a systematic literature review. J Am Geriatr Soc. 2020;68(12):2955–63.
Neff J, et al. Structural competency: curriculum for medical students, residents, and interprofessional teams on the structural factors that produce health disparities. MedEdPORTAL. 2020;16:10888.
Anderson RT, et al. Issues of aging and adherence to health interventions. Controlled Clin Trials. 2000;21(5, Supplement 1):S171–83.
Nishimura A, et al. Improving understanding in the research informed consent process: a systematic review of 54 interventions tested in randomized control trials. BMC Med Ethics. 2013;14(1):28.
Treweek S, et al. Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis. BMJ Open. 2013;3(2):e002360.
Gormley G. Summative OSCEs in undergraduate medical education. Ulst Med J. 2011;80(3):127–32.
Bowen R, et al. A simulation based difficult conversations intervention for neonatal intensive care unit nurse practitioners: a randomized controlled trial. PLoS ONE. 2020;15(3):e0229895.
Qureshi AA, Zehra T. Simulated patient’s feedback to improve communication skills of clerkship students. BMC Med Educ. 2020;20(1):15.
Stevens DL, et al. Medical students retain pain assessment and management skills long after an experiential curriculum: a controlled study. Pain. 2009;145(3):319–24.
Ludwig AB, et al. Group observed structured encounter (GOSCE) for third-year medical students improves self-assessment of clinical communication. Med Teach. 2017;39(9):931–5.
Ludwig A, et al. Four-station group observed structured clinical encounter for formative assessment of communication skills for internal medicine clerks. MedEdPORTAL. 2016;12:10444.
Elliott S, et al. A comprehensive systematic review of the use of simulation in the continuing education and training of qualified medical, nursing and midwifery staff. JBI Libr Syst Rev. 2011;9(17):538–87.
Motoya I, et al. The characteristics of the clinical competence of novice physical therapists and occupational therapists at the time of hiring in a single institution in Japan. Health Prof Educ. 2020;6(4):538–51.
Rushforth HE. Objective structured clinical examination (OSCE): review of literature and implications for nursing education. Nurse Educ Today. 2007;27(5):481–90.
Nnaji C, et al. An innovative simulation-based community-engaged intervention for training research assistants in culturally appropriate informed consent. Prog Community Health Partnersh. 2018;12(3):247–62.
Zabar SB. Angela; Hanley, Kathleen; Kachur, Elizabeth Krajic, Objective structured clinical examinations: 10 steps to planning and implementing OSCEs and other standardized patient exercises. New York: Springer; 2013. p. 45–92.
Van Nuland M, et al. Optimizing the utility of communication OSCEs: Omit station-specific checklists and provide students with narrative feedback. Patient Educ Couns. 2012;88(1):106–12.
Konopasek L, et al. The Group Objective Structured Clinical Experience: building communication skills in the clinical reasoning context. Patient Educ Couns. 2014;96(1):79–85.
Biran LA. Self-assessment and learning through GOSCE (group objective structured clinical examination). Med Educ. 1991;25(6):475–9.
Poore JA, Cullen DL, Schaar GL. Simulation-Based Interprofessional Education Guided by Kolb’s Experiential Learning Theory. Clin Simul Nurs. 2014;10(5):e241–7.
Hanley K, et al. Monitoring communication skills progress of medical students: establishing a baseline has value, predicting the future is difficult. Patient Educ Couns. 2019;102(2):309–15.
Smidt A, et al. The Kirkpatrick model: a useful tool for evaluating training outcomes. J Intellect Dev Disabil. 2009;34(3):266–74.
Bhanji F, et al. The retrospective pre-post: a practical method to evaluate learning from an educational program. Acad Emerg Med. 2012;19(2):189–94.
Quinn SC, et al. Improving informed consent with minority participants: results from researcher and community surveys. J Empir Res Hum Res Ethics. 2012;7(5):44–55.
Kimmick GG, et al. Improving accrual of older persons to cancer treatment trials: a randomized trial comparing an educational intervention with standard information: CALGB 360001. J Clin Oncol. 2005;23(10):2201–7.
Thank you to NIH/NIA for funding this work through the grant NYU Engagement in Longevity and Medicine (ELM) Research Collaborative (1R24AG063725). Thank you to community partners CaringKind and Aging in New York Fund. Thank you to the NYSIM, a public private partnership of NYU Grossman School of Medicine and City University of New York for their support in coordinating this GOSCE.
No conflicts of interest.
Thank you to NIH/NIA for funding this work through the grant NYU Engagement in Longevity and Medicine (ELM) Research Collaborative (1R24AG063725).
Ethics approval and consent to participate
This is a de-identified quality improvement project that learners were invited to participate in. Because of this it was exempt from The New York University Grossman School of Medicine Institutional Review Board (IRB) approval by the NYU Langone IRB policies. NYU Langone is a large academic medical center. As such, the protocol was approved by the NYU Langone IRB. All methods were carried out in accordance with relevant guidelines and regulations.
Participants had no affiliation with the GOSCE organizers. They received an email invitation to opt-in to participate in the session.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Fisher, H., Zabar, S., Chodosh, J. et al. A novel simulation-based approach to training for recruitment of older adults to clinical trials. BMC Med Res Methodol 22, 180 (2022). https://doi.org/10.1186/s12874-022-01643-4
- Clinical Trials
- Older Adults
- Informed Consent