Comparison of sample characteristics in two pregnancy cohorts: community-based versus population-based recruitment methods
BMC Medical Research Methodology volume 13, Article number: 149 (2013)
One of the biggest challenges for population health studies is the recruitment of participants. Questions that investigators have asked are “who volunteers for studies?” and “does recruitment method influence characteristics of the samples?” The purpose of this paper was to compare sample characteristics of two unrelated pregnancy cohort studies taking place in the same city, in the same time period, that employed different recruitment strategies, as well as to compare the characteristics of both cohorts to provincial and national statistics derived from the Maternity Experiences Survey (MES).
One pregnancy cohort used community-based recruitment (e.g. posters, pamphlets, interviews with community media and face-to-face recruitment in maternity clinics); the second pregnancy cohort used both community-based and population-based (a centralized system identifying pregnant women undergoing routine laboratory testing) strategies.
The pregnancy cohorts differed in education, income, ethnicity, and foreign-born status (p < 0.01), but were similar for maternal age, BMI, and marital status. Compared to the MES, the lowest age, education, and income groups were under-represented, and the cohorts were more likely to be primiparous.
The findings suggest that non-stratified strategies for recruitment of participants will not necessarily result in samples that reflect the general population, but can reflect the target population of interest. Attracting and retaining young, low resource women into urban studies about pregnancy may require alternate and innovative approaches.
Recruitment of participants is often one of the biggest challenges for population health studies, regardless of study purpose, design, or outcome. The relative success of multiple types of recruitment strategies has been previously assessed. Webster and colleagues evaluated the recruitment techniques used in a pregnancy study and found a combination of active (e.g., advertising) and passive (e.g., word of mouth) techniques to be effective . However, they also stated that the resultant study sample was less ethnically diverse, more affluent, and more educated than the population of their catchment area, and thus additional methods would be required in the future to obtain a more representative sample . Patterson and colleagues provided a framework for the design and implementation of successful recruitment activities that helped recruiters increase access to the target population and foster negotiating skills . Sanders and colleagues emphasized the importance of comprehensive recruitment programs with multiple strategies employed simultaneously, combined with ongoing assessment of the success of each strategy .
For observational studies such as longitudinal cohorts, a major challenge is recruiting participants who are representative of the target population, so that study findings can be generalized to the population of interest. Golding and Birmingham described lessons learned from previous cohort studies such as the Danish National Birth Cohort study, the Norwegian MoBa study, the Generation R study, and The Avon Longitudinal Study of Parents and Children . Some of the recommendations they listed were using personal contact at enrolment, ensuring recruitment personnel are appropriate and engaged in the study, and using positive enthusiastic promotional material . Strategies worthy of investment from the outset include raising the profile of the study and ensuring clear, comprehensive, and confidential communication with potential participants .
With sampling and subject selection issues considered, questions that arise are “who volunteers for studies?” and “does recruitment method affect composition of the samples?” There are a number of well-established cohorts world-wide, from Great Britain to Australia, China to the U.S. However, few studies have reported on how much the cohort sample reflected the population from which the samples are drawn. One study that did so was the Generation R study, which compared their sample to the Rotterdam population using variables established by Statistics Netherlands . The investigators found that overall the Generation R participants tended to have a higher socio-economic status (SES) than the general population . Thus, the research results may not be generalizable to the population at large, and caution should be taken with interpreting outcomes.
Recently, two cohort studies recruited pregnant women within the same urban centre (Calgary, AB, a city of approximately one million people), and around the same period of time (2008 – 2012): the Alberta Pregnancy Outcomes and Nutrition (APrON) study (http://www.apronstudy.ca) and the All Our Babies (AOB) study. The two studies used different methods of recruitment, which enabled us to compare the strategies that were implemented within the same context. APrON’s method was entirely community-based: pregnant women were approached in maternity and radiology clinics, local businesses, community events, and through city-wide media coverage. The AOB study initially employed a community-based recruitment strategy, and then changed to a population-based strategy in collaboration with the city’s clinical laboratory services, which effectively gave the study access to all pregnant women in Calgary who presented to a physician for medical care. The AOB and APrON community-based strategies were not identical which permitted further comparison between types of community-based recruitment, in addition to comparing community-based to population-based recruitment strategies.
The purpose of this paper was to determine the extent to which different recruitment strategies in two unrelated pregnancy cohort studies taking place in the same city in the same time period affected the resulting sample characteristics. Comparisons were made between the samples, as well as to provincial and national statistics derived from the Maternity Experiences Survey (MES)  (see description of MES under “Methods”). The profile of the MES served as a standard, or ‘target population’ of women and families having children in Alberta and Canada. The specific objectives were to:
Describe the recruitment strategies in the APrON and AOB cohort studies;
Compare the sociodemographic characteristics of participants in the two studies;
Compare the sample characteristics of APrON and AOB to the MES provincial and national survey samples.
About the cohort studies
APrON study. The APrON study is a prospective pregnancy cohort study whose primary focus is to investigate the role of the intrauterine nutrient environment. The primary questions addressed by the APrON study relate to prenatal maternal nutrition as a predictor of maternal mental health (depression and anxiety), birth outcomes (prematurity, congenital anomalies), and long-term child development (neurodevelopment, behaviour, cognitive health).
AOB study. The AOB study is also a prospective pregnancy cohort study, whose goal is to examine maternal well-being during the perinatal period and infant outcomes such as preterm birth, and to identify the current barriers and facilitators to accessing prenatal care in Calgary. The AOB study is following maternal-infant pairs across the early life course to further examine risk and protective factors important for child developmental outcomes and maternal well-being.
Recruitment strategy for APrON
APrON started recruiting in the spring of 2009, and continued until July 2012. Recruitment initially took place in the two major city centres of Alberta (Calgary and Edmonton), but by 2011 about 85% of the sample was from Calgary, so resources for subsequent recruitment were redirected to only Calgary. Recruitment methods differed a little between the two cities, but were consistently community-based. As AOB recruited only in Calgary, the APrON recruitment strategies discussed in this article are based on the activities that took place in Calgary only.
APrON formulated a comprehensive community-based plan to recruit participants . First, high volume maternity clinics were identified and contacted. Research assistants (RAs) were physically present in the waiting areas of clinics that agreed to participate, and approached pregnant women about the study. Radiology clinics were then added as sites for recruitment, also using RAs. At some clinics, nurses recruited on behalf of APrON and were paid $10 per referral. A Public Relations Coordinator was hired to attend community events (e.g., wedding fairs, baby fairs, community festivals), and to negotiate with local businesses to have APrON posters and brochures on display. Multiple APrON investigators were interviewed and featured in newspapers and on television, which attracted attention to the website and project. Although recruitment began in a relatively high socioeconomic (SES) area of the city, attempts to diversify the demographic make-up of the sample resulted in expansion to quadrants of the city with higher proportions of lower-income and new Canadians. To make it easier for women from immigrant/refugee backgrounds to be included in the study, and not have to travel the distance to the primary APrON office, satellite clinics were set up for data collection (including collection of biological specimens) in several physicians’ offices. APrON also made use of social media by setting up Facebook and Twitter accounts.
Recruitment strategy for AOB study
There were two recruitment phases in the AOB study, the first beginning in 2008 and the second in 2009. The objectives of the first phase (which is referred to as the Observational Cohort (OC)) were to examine health care utilization and maternal well-being across the perinatal period. For the second phase (which is referred to as the Prediction Cohort (PC)) the focus expanded to examine biological and environmental markers for spontaneous preterm delivery. Women were recruited to the OC from health care offices, using community posters, word-of-mouth, and through the regional health services website. Women were recruited to the PC using a collaborative strategy with the laboratory service. In Alberta, clinical practice guidelines for prenatal care stipulate viral serology testing for all pregnant women by public health laboratories. In Calgary, this service is provided by Calgary Laboratory Service (CLS), whose lists for prenatal viral serology tests are continually updated as newly pregnant women enter into prenatal care. All women who received prenatal viral serology testing in Calgary during study recruitment (2009–2011) were initially contacted by CLS, who asked permission to release patient contact information to the AOB research staff . Women who consented were telephoned by an AOB research assistant to determine their eligibility.
Provincial and National Statistics from the Maternity Experiences Survey
The Maternity Experiences Survey (MES) was the first of its kind to examine the pregnancy, labour, birth and postpartum experiences of Canadian women. The survey was sponsored by the Canadian Perinatal Surveillance System of the Public Health Agency of Canada (PHAC) (see http://www.phac-aspc.gc.ca/rhs-ssg/pdf/survey-eng.pdf) .
The MES was a cross sectional sample survey, using post-census data from the 2006 Canadian Census of Population to identify babies born between the target dates, as well as the mothers of those babies. The MES sample was stratified by province or territory, and the mother’s age. The sample frame was further refined in some provinces by mother’s residence in a census metropolitan area, or the presence of other children in the household. Mothers aged less than 20 years at the time of their babies’ birth were oversampled. A simple random sample was selected without replacement within each stratum. The final sample consisted of 8,542 women from across Canada, inclusive of all provinces and territories, 6,421 of whom had complete questionnaire information for analyses. Data was collected during the period of October 23, 2006 to January 31, 2007 .
Cross tabulations were performed using Chi2 tests to compare the characteristics of the APrON cohort with AOB-OC, APrON with AOB-PC, and AOB-OC with AOB-PC. Due to multiple comparisons, we set the criteria for statistical significance at p < 0.01 for the Calgary cohort comparisons.
The number of APrON participants was 2200 when recruitment ended in July 2012; the number of AOB participants was 3300 when recruitment ceased in 2011. As not all data are entered at the time of this writing, the sample sizes available for analysis in this paper were APrON = 1200, AOB-OC = 1118, and AOB-PC = 1878. These samples were non-overlapping; i.e., women in the AOB-PC sample were not represented in the AOB-OC sample. APrON and AOB-OC share some similarity in recruitment strategies, namely in-person contact and use of posters. However, APrON’s community based strategy was more extensive, utilizing multiple sources and media types, while AOB-OC focused mainly on perinatal care clinics (Table 1). The inclusion criteria for both APrON and AOB were similar (Table 2). Table 3 summarizes the maternal sociodemographic characteristics of the APrON and AOB cohorts compared to the MES provincial and national pregnancy samples.
Calgary cohort comparisons
The test results were mixed for the various group comparisons (see Table 3). The omnibus Chi 2 p-values and F test p-values for categorical and continuous variables, respectively, were ≤ 0.01 for all comparisons with the exception of maternal age. Pairwise comparisons showed that the women in APrON and AOB-OC were similar for age and parity, but differed for education levels, income, BMI, marital status, being born in Canada, and ethnicity (p < 0.01). Compared to women in AOB-OC, women in APrON were more likely to be married, have higher education and income levels, be Canadian born and Caucasian, and have higher pre/post pregnancy BMI.
Compared to AOB-PC, APrON participants were more likely to have higher education and income levels, be primiparous, and significantly less likely to be born in Canada and non-Caucasian (p < 0.01). The two cohorts were similar in age, marital status and BMI profile.
Women in AOB-OC were more likely to have higher education levels, be non-Caucasian, and have higher pre-pregnancy and postpartum BMIs than women in AOB-PC (p < 0.01).
Target population comparisons
In general, the proportions for the various characteristics of the cohorts were not similar to the data from the MES (see Table 3), indicating that the recruitment strategies used by the pregnancy cohorts did not replicate the stratified approach used by the MES. Compared to the MES, the lowest age, education, and income groups were under-represented, and the cohorts were more likely to be primiparous. Nevertheless, comparisons with other data sources at the local and provincial level such as administrative data on perinatal health and census community profiles during or close to the study time period suggest that the APrON and AOB participants are generally representative of the pregnancy and parenting population at the local (city) and provincial levels. For example, the average age of women in Calgary and Alberta giving birth in 2010 was 30.8 and 29.5 years, respectively . In the Calgary cohorts, the average age at delivery was approximately 30 years. Approximately one-quarter of women in Calgary were foreign-born and one-quarter were a visible minority according to the Canadian Census , with only slightly lower percentages seen in the Calgary cohorts (Table 3). Furthermore, approximately 50% of women in the APrON and AOB studies reported a household income of over 100 K, which aligns with the median income of couple families according to recent statistics from Statistics Canada for 2010 (approximately 97 K) .
This study compared the characteristics of women who were recruited for pregnancy cohorts in the same city during the same time period but using different recruitment strategies. Comparisons were also made to a national study that used a stratified sampling frame and to provincial and local birth statistics during the study time period. The differences in the characteristics of women recruited in the Calgary cohorts were associated with recruitment approach. We cannot rule out the possibility that the content of the research projects held different appeal for different women and may have contributed to sample differences. In general, these findings suggest that even with different recruitment methods, women who participate in longitudinal projects on pregnancy tend to reflect the middle part of the bell curve of the population from which they are drawn, in that they are close to the mean for maternal age, and reflect median levels of education and income. Indeed, the proportions of educated and affluent women were high in all three Calgary cohorts. The AOB-OC cohort had the highest proportion of non-Caucasians, which is a direct reflection of the neighborhood in which community-based recruitment began, specifically the ethnically diverse northeast quadrant of the city which comprises immigrants and people belonging to many cultural groups.
Although the Calgary cohorts were more similar than different in demographic profile, they did not capture the degree of variability seen in the MES sample, particularly characteristics of women having children in rural communities or smaller urban centres, which, as noted above, likely reflects the non-stratified sampling approach of the Calgary cohorts compared to the Census sampling strategy employed by the MES. The MES assured its external validity by oversampling women less than 20 years of age. Comparison with the MES suggests that generalizability of results to some strata of childbearing women (e.g., women under 20) may be limited in the Calgary cohorts. Nevertheless, the Calgary cohorts achieved representativeness of the community/population to which they intend to generalize results, namely pregnant women and families with young children living in Canadian urban centres.
It is worth noting that within the characteristics of both the APrON and AOB cohorts, variability of predictor and outcome variables is not synonymous with variability in sociodemographic background . Despite some overrepresentation of higher SES, there was wide variability within the predictors and outcomes, which did not affect each study’s ability to address its research (outcome) questions : in the case of APrON, the relationship between nutrition and maternal mental health, or in the case of AOB, factors related to prenatal care or adverse birth outcomes. This suggests that the research questions under investigation do not have socioeconomic boundaries, akin to the robust associations found in the well-known Adverse Childhood Experiences (ACE) study among a relatively affluent sample of over 17,000 Americans . Results from the ACE study showed that in a middle-class sample, adverse childhood experiences are common and are a prime determinant of long-term health status .
Comparisons between independently constructed cohorts have inherent limitations. First, the community based recruitment activities for APrON and AOB were disproportionate (i.e. APrON had more activities than AOB); therefore, some of the differences between the two Alberta cohorts are to be expected. Second, the selection criteria differed for APrON and AOB which may have emphasized different characteristics within the respective cohorts. Third, because our analyses are by definition post hoc, we faced the challenge of comparing variables that had been collected in slightly different ways. Despite these limitations, the sampling methods used to recruit our cohorts are sufficient for providing insight into risk associations and are likely the most feasible approach for many pregnancy cohort studies . Nevertheless, future cohort studies that adopt a non-stratified sampling approach would benefit from strategies that enhance diversity in participant characteristics to ensure external validity.
The recruitment strategies of the APrON and AOB studies reflect the importance of commitment in the planning phases for high response rates and low attrition rates. While preliminary comparisons between continuers and discontinuers in the AOB study suggest that attrition was related to lower socioeconomic status and poorer mental health, which is consistent with other cohort studies [4, 16, 17], no difference was found in the APrON study. The recruitment strategies used in both APrON and AOB were diverse and comprehensive, which were congruent with methods discussed in the literature (e.g. word of mouth, promotional materials, personal contact, recruitment sites and locations of target population) [1, 3], in addition to using a city-wide public health laboratory service provider. Further strategies were used for cohort retention and ongoing engagement including newsletters and incentives for participation (APrON and AOB), as well as social media links and annual meetings for participants known as “baby parties” (APrON). Both cohorts ensured that research staff members were well-trained and maintained high research standards. These steps build confidence and trust between study participants and study staff. The effectiveness of different types of recruitment strategies (e.g., word of mouth, local media, clinic-based) in the APrON study has been published elsewhere .
In summary, the results of these analyses underscore the importance of sampling approach in study design for both internal and external validity considerations. Our results show that community and city-wide (i.e., laboratory) approaches can result in sample sizes that are large enough to allow for subanalysis of population characteristics (e.g., parity, ethnicity), and samples that are similar to the urban landscape where the studies were established. However, researchers would do well to specifically target vulnerable groups who tend to be under-represented in research in general. Thus approaches for future studies attempting to correct the under-representation of vulnerable groups might consider focusing on recruitment and retention of women who are marginalized in terms of age, language, income, and education. Specialized strategies will be required to meet the needs of these women.
Webster GM, Teschke K, Janssen PA: Recruitment of healthy first-trimester pregnant women: lessons from the Chemicals, Health & Pregnancy Study (CHirP). Matern Child Health J. 2012, 16: 430-438. 10.1007/s10995-010-0739-8.
Patterson S, Mairs H, Borschmann R: Successful recruitment to trials: a phased approach to opening gates and building bridges. BMC Med Res Methodol. 2011, 11: 73-10.1186/1471-2288-11-73.
Sanders KM, Stuart AL, Merriman EN, Read ML, Kotowicz MA, Young D, et al: Trials and tribulations of recruiting 2,000 older women onto a clinical trial investigating falls and fractures: Vital D study. BMC Med Res Methodol. 2009, 9: 78-10.1186/1471-2288-9-78.
Golding J, Birmingham K: Enrollment and response rates in a longitudinal birth cohort. Paediatr Perinat Epidemiol. 2009, 23 (Suppl 1): 73-85.
Jaddoe VW, Mackenbach JP, Moll HA: The generation R study: design and cohort profile. Eur J Epidemiol. 2006, 21: 475-484. 10.1007/s10654-006-9022-0.
Public Health Agency of Canada: What Mothers Say: The Canadian Maternity Experiences Survey. 2009, Ottawa: Government of Canada
Manca DP, O’Beirne M, Lightbody T, Johnston DW, Dymianiw D, Nastalska K, et al: The most effective strategy for recruiting a pregnancy cohort: a tale of two cities. BMC Pregnancy Childbirth. 2013, 13: 75-10.1186/1471-2393-13-75.
Gracie SK, Lyon AW, Kehler HL, Pennell CE, Dolan SM, McNeil DA, et al: All Our Babies Cohort Study: recruitment of a cohort to predict women at risk of preterm birth through the examination of gene expression profiles and the environment. BMC Pregnancy Childbirth. 2010, 10: 87-10.1186/1471-2393-10-87.
Alberta Reproductive Health Report Working Group: Alberta Reproductive Health: Pregnancies and Births Table Update 2011. 2011, Edmonton, AB: Government of Alberta
Statistics Canada. Calgary, Alberta (Code4806016) (table). 2006: Community Profiles. 2006 Census. Statistics Canada Catalogue no. 92-591-XWE. 2007, Ottawa, ON: Health Canada
Statistics Canada: Family income and income of individuals, related variable: Sub-provincial data, 2010. Statistics Canada Catalogue no. 11-001-X. 2012, Ottawa, ON: Health Canada
Golding J, Jones R, Brune MN, Pronczuk J: Why carry out a longitudinal birth survey?. Paediatr Perinat Epidemiol. 2009, 23 (Suppl 1): 1-14.
VanderWeele TJ: Bias formulas for sensitivity analysis for direct and indirect effects. Epidemiology. 2010, 21 (4): 540-551. 10.1097/EDE.0b013e3181df191c.
Hawkes D, Plewis I: Modelling non-response in the National Child Development Study. J R Statis Soc. 2006, 169: 479-491. 10.1111/j.1467-985X.2006.00401.x.
Manolio TA, Weis BK, Cowie CC, et al: New models for large prospective studies: is there a better way?. Am J Epidemiol. 2012, 175 (9): 859-866. 10.1093/aje/kwr453.
Cartwright A: Who responds to postal questionnaires?. J Epidemiol Community Health. 1986, 40: 267-273. 10.1136/jech.40.3.267.
Hapgood C, Elkind GS: Refusal to participate: effects on sample selection in a longitudinal study of postnatal mood. J Psychosom Obstet Gynaecol (Suppl ). 1989, 10: 89-97. 10.3109/01674828909016681.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/13/149/prepub
APrON would like to thank our study participants for their time and dedication to this research, and to the many prenatal clinics and obstetrical offices that supported recruitment. We also thank Alberta Innovates-Health Solutions for funding, and for the partnerships and support of the following: The Alberta Children’s Hospital Research Institute, The Women and Children’s Health Research Institute, Alberta Health Services, The Alberta Institute for Human Nutrition, The APrON Scientific Advisors, and all study staff in Calgary and Edmonton (especially Dayna-Lynn Dymianiw and Sarah Loehr for assisting with study protocol development).
AOB are extremely grateful to all the families who took part in this study and the whole All Our Babies team. We thank the investigators, co-ordinators, research assistants, graduate and undergraduate students, volunteers, clerical staff and managers. Three Cheers for the Early Years, Alberta Health Services, and the Alberta Children’s Hospital Foundation have provided support for the study. The University of Calgary has provided trainee salary support. Alberta Innovates Health Solutions provided funding for this cohort and salary support for Suzanne Tough.
The authors declare that they have no competing interests.
BL and SM wrote the initial draft of the manuscript, and revised various drafts to produce the completed draft. SM performed the statistical tests for comparing the characteristics of the cohorts. BK and GG collaborated in developing the framework and analyses for the manuscript. BK, GG, and ST reviewed and edited the manuscript. All co-authors participated in manuscript preparation. All authors read and approved the final manuscript.
About this article
Cite this article
Leung, B.M., McDonald, S.W., Kaplan, B.J. et al. Comparison of sample characteristics in two pregnancy cohorts: community-based versus population-based recruitment methods. BMC Med Res Methodol 13, 149 (2013). https://doi.org/10.1186/1471-2288-13-149