- Research article
- Open Access
You are how you recruit: a cohort and randomized controlled trial of recruitment strategies
BMC Medical Research Methodology volume 14, Article number: 111 (2014)
Recruitment is a challenge in developing population-representative pregnancy and birth cohorts.
We developed a collaborative recruitment infrastructure (CRI) to recruit pregnant women for 4 pregnancy cohorts using: faxes from obstetrical offices, in-clinic recruiters, university and funder-driven free-media events, paid-media, and attendance at relevant tradeshows. Recruitment rates and demographic differences were compared between recruitment methods.
We received 5008 referrals over 40 months. Compared to fax, free-media referrals were 13 times more likely to be recruited (OR 13.0, 95% CI 4.2, 40.4: p < 0.001) and paid-media referrals were 4 times more likely to be recruited (OR 4.6, 95% CI 2.1, 10.3: p < 0.001). Among paid-media advertisements, free-to-read print (e.g. Metro) was the most effective (OR 3.3, 95% CI 2.3, 4.5: p < 0.05). Several demographic differences were identified between recruitment methods and against a reference population. Between recruitment methods, media recruits had a similar proportion families with incomes ≥ $40,000 (paid-media: 94.4%; free-media: 93.3%) compared to fax recruits (95.7%), while in-clinic recruits were less likely to have family incomes ≥ $40,000 (88.8%, p < 0.05). Maternal recruits from fax and in-clinic were more likely to attend university (Fax: 92.6%, in-clinic 89.8%) versus the reference population (52.0%; p < 0.05 for both) and both were less likely to smoke (Fax: 6.8%, in-clinic 4.2%) versus reference (18.6%; p < 0.05 for both). However, while fax referrals were more likely to be Caucasian (85.9% versus reference 77.5%; p < 0.05), in-clinic referrals were not significantly different (78.2%; P > 0.05).
Recruitment methods result in different recruitment rates and participant demographics. A variety of methods are required to recruit a generalizable sample.
There is a renewed interest in large, population-representative pregnancy and birth cohort studies. The UK government recently announced a 90,000 longitudinal pregnancy/birth cohort project . The USA National Children’s Study (NCS) plans to recruit 100,000 children . One of the biggest challenges in developing these studies is participant recruitment [3, 4].
Although the NCS originally proposed a pre-conception cohort and estimated that 10–40 households would need to be approached for every enrolled birth, the yield has been markedly lower: 163 households/pregnant women recruited. The NCS is now developing a more traditional pregnancy/birth cohort with recruitment from prenatal care sites . Cohort studies such as the Right from The Start Study (RFTS)  and Avon Longitudinal Study of Parents and Children (ALSPAC) Study  highlight the need for a multifaceted recruitment approach (i.e. community recruitment, prenatal clinics, advertising) with cooperation from collaborators. The NCS’s recently proposed using three recruitment strategies: hospitals and birthing centres, physician/provider referrals, and targeted recruitment for participants of particular scientific interest (i.e. those affected by health disparities, lack of health care access). The NCS intends to recruit a fixed number of participants from each strategy (45000/45000/10000) . We hypothesized that the different recruitment methods will have different recruitment rates, and each recruitment method will approach and recruit a slightly different demographic of participants. Between 2008 and 2012, there were four pregnancy cohort studies recruiting at the University of Alberta. We developed a collaborative recruitment infrastructure (CRI) that recruited for all four studies simultaneously. In this paper we present the results of the CRI.
Description of participating studies
Alberta Pregnancy Outcomes and Nutrition (APrON)
APrON is a study involving thousands of women from Alberta designed to analyze the relationship between maternal nutrient status during pregnancy, maternal mental health, and child health and development. The purpose of APrON is to determine the impact of maternal nutrient intake and status on their own mental health and their children’s neurodevelopment and mental health http://www.apronstudy.ca.
Canadian Healthy Infant and Longitudinal Development (CHILD) study
CHILD is a pan-Canadian longitudinal birth cohort study of 3500 children with follow-up until five years of age. The purpose of the CHILD study is to determine what aspects of the environment interact with genetic factors to affect children’s health and development with a focus on the development of atopic diseases such as asthma. The study includes multiple health and environmental assessments at frequent intervals throughout the study http://www.canadianchildstudy.ca.
Maternal-Infant Research on Environmental Chemicals (MIREC)
MIREC aims to recruit approximately 2000 women from 10 sites across Canada. There are 3 main objectives for MIREC: to measure the extent to which pregnant women and their infants are exposed to chemicals; to measure some of the beneficial elements in human breast milk; and to assess what health risks, if any, are associated with the chemical levels measured, with a focus on heavy metals such as lead and mercury. http://www.mirec-canada.ca.
Trauma in Pregnancy study (TIPS)
TIPS investigates the effects of subtle stressors, as well as of more severe, obvious traumas, on child development and maternal health. Participants are divided into a case study group of 120 women who have experienced trauma during pregnancy and a control group of 120 women whose pregnancies are low risk and trauma free. Mothers are assessed every two months postpartum, and babies are assessed at 6, 12, and 18 months after delivery.
The University of Alberta Health Research Ethics Board (HREB) approved the CRI globally and for each participating study separately. The general procedure for recruitment is as follows. Prospective participants had nominal information collected to determine their eligibility for recruiting studies. Study specific Research Assistants (RAs) contacted the prospective participants, now called referrals, to determine their interest in participanting in one of the studies. If a referral was interested, they would be consented to their respective study. After consenting, the referrals were called recruits. Written informed consent was obtained from the participants for the publication of this manuscript and any accompying images. A copy of written consents are available for review by the Editor of this journal.
The CRI used six methods to obtain referrals and recruit pregnant women. Initially we used a single pamphlet to present all studies for the fax and free-media methods. We subsequently chose to distribute study specific pamphlets on a rotating basis.
Fax: The fax method was developed around the observation that study endorsement by a patient’s health-care provider is an important determinant of recruitment [9, 10]. During pre-natal visits, the clinic’s front desk staff would provide a study pamphlet for the patient to read while waiting for her appointment. The clinic staff (physicians, mid-wives, nurses) would then take up to 1 minute to present the study(s) and ask their patient whether she would consent to having her contact information faxed to the CRI office (fax sheet: Figure 1). Faxes were distributed to the studies that were concurrently recruiting based on nominal inclusion and exclusion criteria. RAs subsequently contacted the women to determine if she would be willing to participate. We remunerated each group practice, minimum 3 physicians with a focus on obstetrics, participating in the fax or in-clinic method (see following) $250/month. We maintained an active dialogue with participating physicians by regular visits and distributing a CRI newsletter.
In-clinic: We stationed RAs in the waiting room of some high volume obstetric clinics. The RAs from the different recruiting studies rotated among the clinics and presented their study to prospective participants. The in-clinic RA had up to 2 minutes to present the study and included some of the personal benefits to participation - the potential for personal benefit is the number one reason why pregnant women participate in research [9, 10].
Tradeshows: APrON and CHILD each had booths at several pregnancy or infant tradeshows. Both studies also held information nights at community centers. Study material was presented in addition to providing study-marketing material such as reusable bags and pens. CHILD subsequently instituted a gift basket draw with ballots available at the tradeshows. The raffle was free and all individuals (pregnant or not) were eligible.
Free-media: Participant interest that resulted from free recruitment methods (i.e. media interest, brochures and poster in locations without RA study endorsement) .
Separate press releases for CHILD and APrON resulted in study coverage in local and national media.
Posters and brochures were placed in locations that provide services for pregnant women. Locations included:
Small volume obstetrics
Family physician clinics
Birth control centers
Pre-natal ultrasound imaging centers
Maternity and baby stores
Direct: Prospective participants would call in to the CRI study office or the study-specific office based on recommendations from friends or other physicians.
Paid-media: Between November 1, 2010 and January 30, 2012, the CHILD study completed a randomized (random blocks) control trial (RCT) to determine the effectiveness of paid-media advertising on study recruitment. Our total advertisement budget was $25 000. The choice of advertisement interventions was determined by discussions with senior principal investigators (who suggested radio), the experience of several senior research coordinators (who suggested transit and free-to-read print), and limited budget (e.g. internet, Facebook, postal). The choice of media outlet (e.g. Sun, Metro, Today’s Parent, CBC.ca, etc.) within each intervention was chosen based on the media outlet demographics. The duration of each advertisement within each media outlet was based on budget considerations in consultation with the media outlet vendor around the optimal duration for the advertisement. Advertisements (e.g. radio script, print advertisement pictures, and logos) were developed by a professional advertisement agency in consultation with the media outlet. Table 1 provides the details for each of advertisement interventions. Advertisement interventions included:
Postal (e.g. Advertising mail-out flyer to a community with a high birth-rate)b. Free-to-Read Print (e.g. Metro, Examiner; (Figure 2))
Internet (e.g. Facebook, CBC.ca, See.ca)
RadioThe order of advertisement intervention was randomly determined (random number). A washout period between interventions of at least 4 times the duration of the intervention was included after each intervention. One of the interventions (Birth Issues) has a 3-month publication cycle. Inclusion of Birth Issues in the RCT would necessitate a 1-year wash-out period. A 1-year wash out would not allow us to complete the advertisement RCT prior to the end of recruitment for the CHILD study. As a result, we elected to advertise in Birth Issues for 1 year during the advertisement RCT (Figure 5).
Tracking referrals through the CRI
A Research Electronic Data Capture: RedCap  database was used to input, assign, and track referral and recruits. The database records basic contact information (name and telephone number) and gestational age (GA) ≤24 weeks. Also, the database logs the date and time of each contact attempt, the results of the attempt (e.g. contacted, answering machine, message left), and any conversation results (e.g. recruited, declined).
We examined the number of calls required by each recruitment method to reach a resolution (recruited, ineligible, or declined.) Generalized linear mixed model with robust errors was used to examine the influence of factors such as such as time of day and day of week of the call on the likelihood of talking to an individual on the phone.
Univariate and multivariate logistic regression (Stata9.2; Stata Corp, College Station, TX) compared the different recruitment methods for the odds of recruiting individuals, while controlling for recruiting study and recruit demographics. Fax recruitment/recruits were used as the reference group because 1) fax recruitment is analogous to the physician referral recruitment strategy most commonly used by research studies, 2) had the largest sample size and 3) fax was the first recruitment strategy utilized.
For both free and paid-media, we analyzed (Poisson regression for count data) for the number of individuals who reported being recruited as a results of a specific advertisement (media-specific recruits) and for all individuals recruited during each advertising intervention as a reflection of increased general awareness of the CHILD study (multi-level Poisson regression controlling for the specific clinics for the fax and in-clinic referrals). We considered a lag between 1 to 4 days in the models (using auto-correlation) to capture the delay between advertising and an individual’s response to the advertisement. For each advertising intervention we included referrals received up to 2 days after the last advertisement as part of the intervention based on the auto-correlation results. We included the presence of Birth Issues advertising in all analyses that included paid-media.
Demographic differences between referrals
Univariate and multivariate logistic and linear regression compared recruitment methods for the age of referrals (calculated from birth date where available), proportion ≥24 weeks GA, ethnicity as determined from last names (general Canadian population, South Asian origin, Chinese origin) , and income quintile (imputed from the 6-digit postal code using the Statistics Canada data Postal Code Conversion File (PCCF+)) .
Demographic differences between recruits
CHILD study data was used to examine differences in the recruit’s demographic data by recruitment method. Univariate and multivariate logistic and linear regression compared recruits by income strata, country of birth, health conditions, mother’s age, and proportion of mothers ≥24 weeks GA at time of recruitment.
The demographics of those recruited to the CHILD study were compared to a reference population of pregnant women identified through a multiple-physician obstetrical practice in Edmonton . Univariate analysis compared recruits (total and by recruitment method) to the reference population for marital status, ethnicity, education, income strata, and health conditions.
The CRI was implemented at sixteen sites across Edmonton (Figure 6). We received 5008 referrals from 10/03/08 to 31/01/12 (Figure 7). There were significant difference in referral rates between fax (reference group) and all other recruitment methods for all studies. We received the most referrals through fax (1.85 referrals/day) and the least from the direct recruitment method (0.05 referrals/day; p < 0.05). Each tradeshow garnered approximately 46 referrals (p < 0.05 versus fax referrals).
From the 3465 referrals we have call data on, individuals were more likely to be spoken to in the evenings (40.3%; p < 0.001; Table 2) compared to mornings (34.0%) and afternoons (35.2%). Individuals were also more likely to be contacted on the weekend (Saturday: (Odd ratio (OR): 1.68; 95% CI 1.06, 2.66; p: 0.03; Sunday: OR 1.52; 95% CI 0.95, 2.42 p: 0.08) compared to weekday calls. Referrals identified through direct recruitment required the fewest calls (Median 1 call, Interquartile range (IQR) 3) to reach resolution (recruited or declined), followed by media (Median 3 calls, IQR 2), in-clinic and fax (Both: median 4 calls; IQR 5), and tradeshows (median 5 calls, IQR 6). Compared to fax, call attempts from all other recruitment methods were statistically significantly different (p < 0.001).
We have recruitment data on 4123 individuals (3169 eligible referrals, 954 ineligible referrals based on study inclusion and exclusion criteria). Of 3169 eligible individuals, 982 individuals (31%) were successfully recruited. Table 3 provides the demographic differences between individuals recruited and non-recruited individuals among those referred. Amongst those recruited, we noted significant differences in recruitment success and participant demographics by recruitment method.
Results for each recruitment method are presented in the following order 1) any significant demographic differences in referrals compared to fax referrals, 2) any significant difference in recruitment rate compared to fax referrals, 3) any significant demographic difference in recruits compared to fax referrals. For all studies, there was no significant difference in the mean age between recruitment methods for referrals or recruits (Table 4). Amongst CHILD Study recruits, there were no significant difference in the recruit demographics between any of the recruitment methods for maternal mean age, maternal attendance at a post-secondary institution, maternal smoking history and paternal history of asthma (Table 5).
In-clinic referrals had a significantly lower mean income quintile (2.9, SD 1.4; n = 1479) compared to fax referrals (3.3, SD = 1.4; n = 2044; p < 0.05 for both). In-clinic referrals continued to have a lower mean income quintile (Table 6) when analyses were restricted to clinics where both fax and in-clinic recruitment methods were used.
Among CHILD study recruits, there was a significant difference in the proportion of married individuals between the fax and in-clinic recruits (Fax: 97.7% married; In-Clinic: 93.4% married; p < 0.05). The proportion of individuals who identified themselves as Caucasians differed between the fax and in-clinic recruits (Fax: 85.9%, In-Clinic: 78.2%, p < 0.05). For all studies, compared to fax recruits, in-clinic recruits were later in pregnancy (Fax: 45.4% ≥ 24 weeks GA; In-Clinic: 54.4% ≥ 24 weeks GA; p < 0.05) and had a significantly lower mean income quintile (Fax: 3.5/5, SD 1.3; n = 290, In-Clinic: 3.0/5, SD 1.4; n = 367; p < 0.05). In-clinic recruits continued to be later in GA and have a lower mean income quintile when analyses were restricted to clinics where both fax and in-clinic recruitment methods were used.
Recruitment success from tradeshows depended on whether a raffle was held. Compared to fax recruits, when a raffle was held, referrals were 44% less likely to be recruited (OR 0.56, 95% CI 0.41, 0.75; p < 0.001; Table 7). When a raffle was not held, referrals were 72% less likely to be recruited compared to fax recruits (OR 0.28, 95% CI 0.14, 0.58, p < 0.001). Analyses controlled for differences between studies, GA at the time of referral, and family income quintiles.
Media (free and paid media)
Referrals from media were earlier in pregnancy (14.7% of women >24 weeks GA) compared fax referrals (58.0% >24 weeks GA; p < 0.05). Free-media referrals were 13 times more likely to be recruited than fax referrals (OR 13.0, 95% CI 4.18, 40.43: p < 0.001), while paid-media referrals were over 4 times more likely to be recruited than fax referrals (OR 4.6, 95% CI 2.08, 10.31; p < 0.001).
For all study recruits, women recruited through media were earlier in pregnancy (11.3% of women >24 weeks GA) compared to fax recruits (45.4% >24 weeks GA; p < 0.05). This statistically significant difference was similar in CHILD study recruits. For CHILD study recruits, media recruits were earlier in pregnancy (Free-media: 35.3% of women >24 weeks GA; Paid-media: 26.9% >24 weeks GA) compared to fax (47.4%, p < 0.05 for both). Among CHILD Study recruits, free-media recruits were more likely to report a history of asthma compared to fax recruits (Free-media: 47.1% versus Fax: 20.3%; p < 0.05). There was no difference in maternal asthma history between paid-media and fax recruits.
Comparison of CHILD Study Recruits to a Reference Population
CHILD study participants recruited from fax and in-clinic were different from each other and from the reference population. CHILD study recruits, overall, were more likely to attend university (Total: 90.9%, Fax: 92.6%, in-clinic 89.8%; Table 8) versus the reference population (52.0%; p < 0.05 for all three); were less likely to smoke (Total: 5.0%, Fax: 6.8%, In-clinic 4.2%) versus reference (18.6%; p < 0.05 for all three); and come from the highest income quartile (Total: 81.6%, fax: 83.9%; in-clinic: 77.9%) versus reference population (42.0%; p < 0.05).
Although the proportion of married individuals among total recruits (94.6%) and the reference population was not statistically significantly different (93.5%; p < 0.05), there were significant differences in marital status between fax recruits and the reference population (97.7% married; p < 0.05). This difference from the reference population was not observed among in-clinic recruits (93.4%; p > 0.05). While total recruits and fax recruits, were more likely to be Caucasian (Total: 82.3%; Fax: 85.9% versus reference 77.5%; p < 0.05 for both), the proportion of Caucasian amongst in-clinic recruits was not significantly different from the reference population (78.2%; p > 0.05). For the recruits that identified themselves of first-nations origin, there were no significant difference between the fax recruits (3.5% identifying themselves as first-nation) compared to the reference population (3.7%, p > 0.05). However, in-clinic recruits has a disproportionate percentage of individuals of first nations origin (6.8%; p < 0.05) when compared to the reference population.
For the RCT analysis, inclusion of referrals 2 days after the advertisement intervention provided the best model fit. Between the free and paid media recruitment methods, referrals differed in mean age, proportion of women >24 weeks GA, and mean income quintile (Table 9). Amongst all paid-media referrals, internet referrals were the oldest (Internet: mean age 31.3 yr., SD 4.3); paid-to-read referrals had the highest proportion of women later in pregnancy (76.2% >24 weeks GA); and transit advertisements had the highest mean income quintile (mean 3.4, SD 1.4). The demographic findings by paid media method were all statistically different compared to free-media referrals (mean age: 30.6 yr, SD 4.8; 45% >24 weeks GA; mean income quintile: 2.8, SD 1.4; all three p < 0.05).
Free-to-read print (e.g. See, Metro) was the most effective method of advertising (OR 3.3, 95% CI 2.34, 4.51; p < 0.05; Table 10) with a cost of $63.44/recruit. The most cost-effective method of recruitment, while still providing a significant recruitment advantage over no-advertisement (reference) was advertising in Birth Issues at a cost of $1.56/recruit (OR 1.97, SD 1.64. 2.37; p < 0.05). Each referral costs the study human resources in an attempt to convert the referral to a recruit. $/OR measurements accounts for recruitment effectiveness (converting a referral into a recruit) whereas a $/recruit measurement only accounts for recruits. A recruitment method that has higher recruit:referral, as captured by an OR, is a more cost effective recruitment method. Based on this metric, Birth Issues was still the most efficient paid-media method ($0.79/OR of recruiting someone), followed by trade-specific publications ($13.27), Internet ($15.00), and free-to-read print ($19.52).
Among paid advertisements with a recruitment rate significantly greater than no advertisement (reference), radio and Birth Issues had the slowest recruitment rate averaging only 1.0 recruits/day and 1.4 recruits/day respectively. Both free-to-read and paid-to-read print had a greater number of recruits per day in total than the other paid-media methods (Free-to-read: 2.61 recruits/day; Paid-to-read: 3.17 recruits/day).
Several demographic differences were noted among the paid and free-media recruits (reference). Recruits from free-media were less likely to have post-secondary education (55.6%) versus recruits from paid-media (range of 89.7% to 100% with post-secondary education; p < 0.05; Table 11). Similarly, free-media recruits were less likely to have a family income ≥ $80,000 (33.3%) versus paid-media recruits (range of 60.7% to 77.3%; p < 0.05). Finally, free-media recruits were less likely to be married (77.8%) when compared to paid-media recruits (Range: 93.6% to 100%; p < 0.05).
When we examined specific demographic differences within paid media, free-to-read print recruits were older (mean age 32.3 years, SD 4.4) while recruits from trade-specific publications (30.9 years, SD 4.5) and Birth Issues (30.9 years, SD 4.5) were younger than free-media recruits (31.5 years, SD 4.6; p < 0.05 for all 3). Free-media recruits were earlier in pregnancy (30.0% >24 weeks GA) compared to most of the paid-media advertisements including the trade-specific publications (55.3% >24 weeks GA; p < 0.05), Birth Issues (53.8%; p < 0.05) and paid-to-read print (72.2%; p < 0.05). Recruits from the trade-specific publications and Birth Issues were less likely to report a prior health condition (22.1% and 22.9% respectively) versus free-media recruits (25.0%; p < 0.05 for both).
Our study identifies that recruitment methods can introduce participant demographic bias, which expands on our understanding that unrepresentative recruitment can result in selection biases . This analysis of recruitment methods for longitudinal pregnancy cohort studies found substantial differences in the recruitment rates between recruitment methods. Participant demographic bias may be due to clinical, social, ethical and ethnic differences [17–19]. Results from the CRI suggest that 1) demographic differences in the individuals referred by each method leads to 2) demographic differences in the individuals recruited by each method which results in, 3) demographic differences in the individuals recruited when compared to a reference population. This trend is most obvious when examining demographic differences in mean income from referrals through to recruits. Multiple recruitment methods are required to overcome this source of participant demographic bias.
Demographic differences in referrals between recruitment methods likely represent a combination of factors including differences in clinic demographics, referral bias, and the level of engagement from front-line clinic staff. Physicians participating in the fax recruitment method present the study information to those patients who they feel would be most likely to join the study with a possible bias to married, higher socio-economic status individuals. The in-clinic recruiters approach all individuals and present the benefits of participating; this presentation may appeal to a wider population. This bias in study presentation is reflected in a lower mean family income and more variable marital status for in-clinic referrals and, consequently, in-clinic recruits. Demographic differences by recruitment method highlight the importance of using multiple recruitment strategies to recruit a population-representative sample.
Recruitment methods that encourage prospective participants to contact our central recruitment office, such as free and paid-media strategies, save RA calls and RA time. Our study showed that respondents to both type of media had the highest rate of recruitment. However, referrals (and recruits) from free-media respondents were more likely to have a health condition associated with the research (e.g. asthma for the CHILD study). This health-condition bias was not observed among the paid-media recruits and, in general, there were few substantive demographic differences between paid-media and fax recruits (reference). The less expensive advertisement methods have a lower recruit/day rate and conversely, the more expensive advertisement methods provide a higher number of recruits/day. Free and paid-media are effective methods of recruiting participants, although a study’s choice of paid-media advertisement will be determined by the study’s budget and recruitment time frame.
Recruiting a sample of pregnant women has required the development of substantial recruitment infrastructure with support from multiple stakeholders. We worked with all four birth cohort local principal investigators, four hospitals, two health regions, multiple physicians (both academic-appointed and those in private-practice), multi-disciplinary healthcare providers (e.g. mid-wives, nurses), health-care groups (laboratory services, ultrasound clinics), and community organizations (baby stores, media organizations, trade show organizers, interest groups) in developing the CRI. Support from all stakeholders is a pre-requisite to developing and using multiple recruitment strategies.
We developed a CRI referral process that met the needs of community-based clinicians. Lack of support from clinicians can present challenges to recruitment . Barriers to staff participation in research include: time [15, 21], understanding the many complicated protocols , a lack of perceived rewards , and interference with the health-care professional/patient relationship [23, 24]. The CRI attempted to overcome many of these barriers. Health-care professionals do not give detailed study explanations nor do they consent participants; this reduces the recruitment burden on their practice with minimal impact on their patient relationship [21, 23]. Distributing prospective research participants across several studies broadens the entry criteria making it more efficient for healthcare professionals to participate in numerous studies with varying inclusion and exclusion criteria. The $250 reimbursement was based on evidence supporting monetary incentives to encourage healthcare professionals to recruit patients . As a result of our work, the fax process provided the greatest number of referrals/day compared to all other recruitment methods including sitting in-clinic.
One of the limitations of this study is the limited demographic data available for all referrals. The choice of information collected from referrals was focused on obtaining the least amount of information necessary to efficiently triage patients to the participating CRI studies. As an example, while some studies excluded individuals on anti-depressant medication, we elected not to put this criterion on the fax recruitment sheet as it would increase the burden for the community-based obstetricians. Similarly, the in-clinic recruiter did not routinely obtain the referral’s date of birth or marital status since this information was not critical for the distribution of prospective participants to participating research studies. The use of postal codes as a surrogate for household income is a result of the necessarily limited data collected on referrals. Although we do not have any detailed recruit demographic data for the non-CHILD studies, we used more detailed demographic data from the CHILD study (over 85% of all CRI recruits) in an attempt to overcome this limitation.
We encountered several difficulties in designing and implementing the advertisement RCT. We had originally proposed to run a single advertisement intervention (i.e. only transit or only radio or only free-to-read print). However, we could not identify any prior publications that identified the most effective advertisement method for health research study recruitment. As a result, we chose to test common advertisement methods. This decision resulted in reduced power when examining the effect of a single advertisement intervention on recruitment (a more definitive study). Some of our media outlet vendors did not run our advertisement, while others did not run the advertisement during the proscribed period and compensated us, without telling us, by running the advertisement at other times or for a longer period of time. We tried to control for this “contamination” through statistical analyses by controlling for the number of studies run on a specific day.
We chose to implement the advertisement RCT within the context of the CHILD study due to fiscal and staffing considerations. Using the CHILD study as the recruiting study resulted in compromises to the advertisement RCT. We originally proposed to test each advertisement intervention twice in order to limit any seasonal bias on referrals and recruits. Unfortunately, the CHILD Study closed recruitment prior to the completion of two advertisement “runs” for each advertisement intervention. We tried to control for the seasonal effect of advertisement in the regression analysis in an attempt to control for this limitation.
One of the strengths of this analysis is the large number of referrals (over 5000). The CRI was implemented at all 4 obstetrical hospitals in Edmonton to ensure a sample of pregnant women that was as population representative as possible. The prospective, systematic capture of referral data ensures minimal recall bias. The systematic, real-time, capture of recruit demographics could be utilized to adjust recruitment strategies to ensure a population-representative sample. We believe the results obtained from this analysis are timely and generalizable to currently recruiting pregnancy and birth cohort studies.
We completed over 16,000 telephone calls as part of the CRI resulting in a significant human resource cost. Future research may examine the success of volunteers, automated phone dialers and/or call centers in recruiting study participants. Future research may also include conducting multiple RCTs of each of the paid advertisement versus a non-advertisement control outside the confines of an active research study to better understand the effectiveness of each of the advertisement methods in medical research.
Successful recruitment requires collaboration and active engagement from all major stakeholders. Implementing a study of recruitment methods into large, population-based studies requires pre-planning and additional study infrastructure. The results of our study help improve our understanding one of the most expensive and time-consuming but often over-looked aspects of study design. A variety of recruitment methods are required to generate a large, population-representative birth-cohort study.
The Arts, Humanities and Social Sciences in the Modern University. [http://www.bis.gov.uk/news/speeches/david-willetts-arts-humanities-social-sciences],
Funding and Legislation. [https://www.nationalchildrensstudy.gov/about/funding/Pages/default.aspx],
Prescott RJ, Counsell CE, Gillespie WJ, Grant AM, Russell IT, Kiauka S, Colthart IR, Ross S, Shepherd SM, Russell D: Factors that limit the quality, number and progress of randomised controlled trials. Health Technol Assess. 1999, 3: 1-143.
Lovato LC, Hill K, Hertert S, Hunninghake DB, Probstfield JL: Recruitment for controlled clinical trials: literature summary and annotated bibliography. Control Clin Trials. 1997, 18: 328-352. 10.1016/S0197-2456(96)00236-X.
Savitz DA, Ness RB: Saving the National Children’s Study. Epidemiology. 2010, 21: 598-601. 10.1097/EDE.0b013e3181e942cc.
Promislow JH, Makarushka CM, Gorman JR, Howards PP, Savitz DA, Hartmann KE: Recruitment for a community-based study of early pregnancy: the Right From The Start study. Paediatr Perinat Epidemiol. 2004, 18: 143-152. 10.1111/j.1365-3016.2003.00546.x.
Golding J, Pembrey M, Jones R, Team AS: ALSPAC–the Avon Longitudinal Study of Parents and Children. I Study methodology. Paediatr Perinat Epidemiol. 2001, 15: 74-87. 10.1046/j.1365-3016.2001.00325.x.
Guttmacher AE, Hirschfeld S, Collins FS: The National Children’s Study--a proposed plan. N Engl J Med. 2013, 369: 1873-1875. 10.1056/NEJMp1311150.
Kenyon S, Dixon-Woods M, Jackson CJ, Windridge K, Pitchforth E: Participating in a trial in a critical situation: a qualitative study in pregnancy. Qual Saf Health Care. 2006, 15: 98-101. 10.1136/qshc.2005.015636.
Mihrshahi S, Vukasin N, Forbes S, Wainwright C, Krause W, Ampon R, Mellis C, Marks G, Peat J: Are you busy for the next 5 years? Recruitment in the Childhood Asthma Prevention Study (CAPS). Respirology. 2002, 7: 147-151. 10.1046/j.1440-1843.2002.00384.x.
McIntosh S, Ossip-Klein DJ, Spada J, Burton K: Recruitment strategies and success in a multi-county smoking cessation study. Nicotine Tob Res. 2000, 2: 281-284. 10.1080/14622200050147556.
Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG: Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009, 42: 377-381. 10.1016/j.jbi.2008.08.010.
Shah BR, Chiu M, Amin S, Ramani M, Sadry S, Tu JV: Surname lists to identify South Asian and Chinese ethnicity from secondary data in Ontario, Canada: a validation study. BMC Med Res Meth. 2010, 10: 42-10.1186/1471-2288-10-42.
Khan S, Wilkins R: Census of Canada. Postal Code Conversion File, PCCF+ Version 5G December 2009 Postal Codes, 2006 . Book Census of Canada. Postal Code Conversion File, PCCF+ Version 5G December 2009 Postal Codes, 2006 . 2010, Statistics Canada. Health Statistics Division, 2010
Simes RJ, Tattersall MH, Coates AS, Raghavan D, Solomon HJ, Smartt H: Randomised comparison of procedures for obtaining informed consent in clinical trials of treatment for cancer. Br Med J (Clin Res Ed). 1986, 293: 1065-1068. 10.1136/bmj.293.6554.1065.
Pastore LM, Dalal P: Recruitment strategies for an acupuncture randomized clinical trial of reproductive age women. Complement Ther Med. 2009, 17: 229-235. 10.1016/j.ctim.2009.03.004.
Al-Shahi R, Vousden C, Warlow C: Bias from requiring explicit consent from all participants in observational research: prospective, population based study. BMJ. 2005, 331: 942-10.1136/bmj.38624.397569.68.
Hille ET, Elbertse L, Gravenhorst JB, Brand R, Verloove-Vanhorick SP: Nonresponse bias in a follow-up study of 19-year-old adolescents born as preterm infants. Pediatrics. 2005, 116: e662-e666. 10.1542/peds.2005-0682.
Rahi JS, Manaras I, Tuomainen H, Lewando Hundt G: Engaging families in health services research on childhood visual impairment: barriers to, and degree and nature of bias in, participation. Br J Ophthalmol. 2004, 88: 782-787. 10.1136/bjo.2003.033258.
Ross S, Grant A, Counsell C, Gillespie W, Russell I, Prescott R: Barriers to participation in randomised controlled trials: a systematic review. J Clin Epidemiol. 1999, 52: 1143-1156. 10.1016/S0895-4356(99)00141-9.
Langley GR, Sutherland HJ, Wong S, Minkin S, Llewellyn-Thomas HA, Till JE: Why are (or are not) patients given the option to enter clinical trials?. Control Clin Trials. 1987, 8: 49-59. 10.1016/0197-2456(87)90025-0.
Williams CJ, Zwitter M: Informed consent in European multicentre randomised clinical trials–are patients really informed?. Eur J Cancer. 1994, 30A: 907-910.
Taylor KM, Margolese RG, Soskolne CL: Physicians’ reasons for not entering eligible patients in a randomized clinical trial of surgery for breast cancer. N Engl J Med. 1984, 310: 1363-1367. 10.1056/NEJM198405243102106.
Taylor KM, Feldstein ML, Skeel RT, Pandya KJ, Ng P, Carbone PP: Fundamental dilemmas of the randomized clinical trial process: results of a survey of the 1,737 Eastern Cooperative Oncology Group investigators. J Clin Oncol. 1994, 12: 1796-1805.
Bryant J, Powell J: Payment to healthcare professionals for patient recruitment to trials: a systematic review. BMJ. 2005, 331: 1377-1378. 10.1136/bmj.331.7529.1377.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/14/111/prepub
The Collaborative Recruitment Infrastructure (CRI) was funded in part by the Women’s and Children’s Health Research Institute (WCHRI) at the University of Alberta (UofA). The APrON study is funded from Alberta Innovates and Health Solutions (AIHS). The CHILD study is funded jointly from the Canadian Institutes of Health Research (CIHR) and the AllerGen NCE. The TIPS study is funded through a grant from the Alberta Centre for Child, Family and Community Research. The MIREC study is funded through Health Canada, The Ontario Ministry of the Environment, and CIHR. Amy Maghera received but declined a studentship from Women’s and Children’s Health Research Institute (WCHRI) for this project.
The authors declare that they have no competing interests.
AM helped develop and expanded the CRI project, implemented the CRI electronic database, and expanded the CRI database to include early referral participant data. PK implemented the advertising RCT. AL helped develop and expanded the CRI project, and implemented the CRI electronic database. YZ helped complete analysis for the CRI project. DM was the investigator in charge of recruitment for the APrON study during first 2 years of the CRI, helped develop the CRI project, and contributed data to the CRI project. TL was the investigator in charge of recruitment for the TIPS study during first 2 years of the CRI, helped develop the CRI project, and contributed data to the CRI project. DH was investigator in charge of recruitment for the MIREC study during first 2 years of the CRI, and contributed data to the CRI project. CH is the Chief of Obstetrics at the Royal Alexandra Hospital in charge of local CRI implementation, helped develop the CRI project, and contributed data to the CRI project. TC is Chief of Obstetrics at the Grey Nuns Hospital in charge of local CRI implementation, helped develop the CRI project, and contributed data to the CRI project. PM was investigator in charge of recruitment for the CHILD study during first 2 years of the CRI, developed and expanded the CRI project, completed statistical analysis, and was primary author of manuscript. All authors contributed to manuscript development, read, and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Authors’ original file for figure 1
Authors’ original file for figure 2
Authors’ original file for figure 3
Authors’ original file for figure 4
Authors’ original file for figure 5
Authors’ original file for figure 6
Authors’ original file for figure 7
About this article
Cite this article
Maghera, A., Kahlke, P., Lau, A. et al. You are how you recruit: a cohort and randomized controlled trial of recruitment strategies. BMC Med Res Methodol 14, 111 (2014). https://doi.org/10.1186/1471-2288-14-111
- Birth cohort
- Research methods
- Sample bias