Skip to main content
  • Research article
  • Open access
  • Published:

Comparing two sampling methods to engage hard-to-reach communities in research priority setting

Abstract

Background

Effective community-partnered and patient-centered outcomes research needs to address community priorities. However, optimal sampling methods to engage stakeholders from hard-to-reach, vulnerable communities to generate research priorities have not been identified.

Methods

In two similar rural, largely Hispanic communities, a community advisory board guided recruitment of stakeholders affected by chronic pain using a different method in each community: 1) snowball sampling, a chain- referral method or 2) purposive sampling to recruit diverse stakeholders. In both communities, three groups of stakeholders attended a series of three facilitated meetings to orient, brainstorm, and prioritize ideas (9 meetings/community). Using mixed methods analysis, we compared stakeholder recruitment and retention as well as priorities from both communities’ stakeholders on mean ratings of their ideas based on importance and feasibility for implementation in their community.

Results

Of 65 eligible stakeholders in one community recruited by snowball sampling, 55 (85 %) consented, 52 (95 %) attended the first meeting, and 36 (65 %) attended all 3 meetings. In the second community, the purposive sampling method was supplemented by convenience sampling to increase recruitment. Of 69 stakeholders recruited by this combined strategy, 62 (90 %) consented, 36 (58 %) attended the first meeting, and 26 (42 %) attended all 3 meetings. Snowball sampling recruited more Hispanics and disabled persons (all P < 0.05). Despite differing recruitment strategies, stakeholders from the two communities identified largely similar ideas for research, focusing on non-pharmacologic interventions for management of chronic pain. Ratings on importance and feasibility for community implementation differed only on the importance of massage services (P = 0.045) which was higher for the purposive/convenience sampling group and for city improvements/transportation services (P = 0.004) which was higher for the snowball sampling group.

Conclusions

In each of the two similar hard-to-reach communities, a community advisory board partnered with researchers to implement a different sampling method to recruit stakeholders. The snowball sampling method achieved greater participation with more Hispanics but also more individuals with disabilities than a purposive-convenience sampling method. However, priorities for research on chronic pain from both stakeholder groups were similar. Although utilizing a snowball sampling method appears to be superior, further research is needed on implementation costs and resources.

Peer Review reports

Background

A key feature of community-based and patient-centered outcomes research is partnering with community stakeholders from a project’s inception to ensure that it offers value to the community, is culturally appropriate, and is likely to yield sustainable improvements in prioritized outcomes [1]. Engaging persons from hard-to-reach or vulnerable communities has high priority, given evidence that lack of engaging racial-ethnic minorities and lower socioeconomic populations in research and decision-making contributes to disparities in enrollment in randomized clinical trials, cancer prevention, and access to evidence-based advances in medicine [25]. Effective strategies to proactively engage and learn from communities experiencing greater health disparities need to inform the development of community-partnered research [6].

Optimal sampling methods to engage community members to elicit ideas and priorities for community-based participatory research continue to be developed [7, 8]. The Methodology Committee of the Patient-Centered Outcomes Research Institute (PCORI) highlighted improving patient engagement methods as one of four high priority areas for standards development [9, 10]. Although groups experiencing the greatest health disparities likely have the greatest “stake” in ensuring that research meets their personal and community’s needs, well-recognized challenges in establishing partnerships between researchers and members from these communities can undermine efforts to address health disparities [5].

Commonly used sampling methods for identification of participants in community settings vary from random to purposive [11, 12]. It might be expected that researchers should randomly sample from among eligible individuals in a community but this approach is resource-intensive and less demanding sampling methods have performed well in eliciting information that reflects broadly-held community beliefs and ideas [10]. Thus, non-probability sampling methods are preferred for recruitment of stakeholders. Table 1 presents the four sampling methods used most often and some of the challenges and benefits of each one. In addition, ethical challenges can arise in implementing strategies to recruit community partners such as misunderstandings about inclusion and exclusion criteria and potential loss of confidentiality and privacy. However, when conducted in close collaboration with community advisors, engaging representative samples of community members who are most affected by a specific issue such as obesity, HIV, and other issues can yield rigorous relevant research programs [1315].

Table 1 Review of four sampling strategies commonly used in community-engaged research

This study was designed to advance understanding of the impact of sampling in community-engaged research by comparing the effectiveness of two non-probability sampling methods to recruit and engage community stakeholders from hard-to-reach, vulnerable populations in identifying priorities for community-partnered research.

Methods

In two similar rural, predominantly Hispanic counties, one of two sampling methods was selected for use in each community: 1) snowball sampling, a chain-referral method where initial participants (seeds) recruit others from their social network or 2) purposive sampling, also known as judgmental, selective or subjective sampling (Fig. 1). Both methods have been used to recruit hard-to-reach subjects for research studies [16, 17] but have not been compared for the purposes of developing research priorities. For this study, we recruited persons affected by chronic non-cancer pain, either personally or as a caregiver, to generate priorities for research on interventions that could improve outcomes of persons affected by chronic pain in their communities. We conducted a series of three facilitated group meetings, using the nominal group technique, to brainstorm, and prioritize ideas regarding services and support [18, 19] for research on chronic pain.

Fig. 1
figure 1

Two Sampling Methods

Our second objective was to compare categories of ideas to improve outcomes of persons with chronic pain generated by participating stakeholders from these two rural, predominantly Hispanic communities. This real-world implementation study not only addresses a priority articulated by PCORI for methods development, it also offers unique insights into priorities for research addressing chronic pain in under-resourced, vulnerable communities.

Setting

For this project, we partnered with residents of two similar rural communities (Frio and Karnes Counties, Texas) that have had limited involvement in research to advance health despite significant health disparities. These communities are rural and predominantly low income Hispanic. The first community, Frio County, has an estimated population of 18,793 with 70 % of residents aged 19–64 years, 41 % women, and 78 % Hispanic ethnicity (of whom 13 % are foreign born) [20]. Frio County has a median household income of $35,681 and 26 % of residents live in poverty while 68 % completed high school and 23 % are medically uninsured. The second community, Karnes County, has 14,975 residents of whom 70 % are aged 19–64, 41 % women and 52 % Hispanic ethnicity (of whom 6 % are foreign born) [21]. The median household income is $44,650 and 22.3 % of residents live in poverty. Of Karnes residents, 19 % are uninsured, and 72 % have completed high school. Among residents of Frio and Karnes Counties, 12 % and 15 %, respectively, are disabled.

Study participants

The topic of chronic pain was selected as the community priority to guide the research because of its significant negative impact on health in the U.S., as highlighted in a report by the Institute of Medicine [22]. Residents of the community in both Frio and Karnes were eligible to serve as stakeholders for group meetings about chronic pain if they were 35–75 years of age and experienced chronic pain for at least three months that negatively affected their daily activities and/or sleep. Potential participants were excluded if their pain was due to cancer because of differing management priorities. We also excluded clinicians such as physicians and nurses because as trained experts they may have strong viewpoints and could dominate discussions with patients and caregivers as seen in other studies [2326]. In addition, caregivers aged 35–75 of persons with chronic non-cancer pain were eligible for participation. In preparation for this project, our research team reviewed several options for non-probability sampling with a standing translational advisory board at our academic institution. Snowball sampling and purposive sampling were selected because the translational advisory board judged them to be likely to yield the desired diverse sample of community members.

Approach

The project first identified a lead community partner in each community. We successfully engaged the director and staff of Texas A&M AgriLife Extension Services in both counties because this organization shares our mission to improve the health of the community through evidence-based advances. Additionally, a member of our team had previously worked with this group on pilot projects. In each county, the AgriLife Extension agent recruited a community advisory board (CAB) to help develop and guide the study by: reviewing all recruitment materials and methods; assisting with stakeholder recruitment; addressing recruitment problems; and leading meetings to disseminate and act on community stakeholders’ ideas.

Sample size

The initial recruitment goal for the study was 130 participants, 65 per sampling method. This goal was based on concept mapping studies where 40 to over 100 participants were involved in idea generation [27].

Snowball sampling approach – community 1

Incentivized snowball sampling uses a modified chain-referral approach where a small number of recruits (seeds) meeting eligibility criteria and after consenting to participate then receive a small incentive for recruiting others from their social network who also meet eligibility criteria [28] (Table 2). Persons recruited by “seeds” identified others with desired characteristics and then those individuals identified others until either the sample size goal was achieved or the timeframe for recruitment ended. This approach has the advantage of efficiency and, when networks are broad, final recruits can be independent of initial recruiters [29, 30]. For our project, a convenience sample of “seeds” were first contacted by a CAB member who used talking points developed by the CAB and team members to solicit interest in participation and offered a flyer about the project as additional information. If agreeable, a project coordinator then contacted the individual to assess for eligibility. Eligible “seed” stakeholders (N = 12) who consented to participate were informed about the project, provided the talking points developed by the CAB, and asked to recruit others who met study inclusion criteria. The “seeds” received a $5 gift certificate for each eligible recruit.

Table 2 Characteristics of stakeholders recruited by sampling method

Purposive to convenience sampling approach – community 2

We initially used purposive sampling that identifies participants from specific constituencies from thorough analysis (or database, if available) of the target community’s characteristics and assets [31] (Table 2). Preparation for purposive sampling involved a review of relevant data about each community by CAB members and researchers [20, 21, 32, 33] leading to the development of a matrix of categories of diverse constituencies and gender/age groups to guide recruitment. We aimed to recruit participants with chronic non-cancer pain from diverse community, social, and work organizations including: businesses, volunteer organizations (local Rotary and Lions clubs), faith-based groups, school districts, and agricultural groups such as local ranchers. However, several groups, especially businesses based outside of the community, refused to participate, citing company policies. After conferring with the CAB from community 2, we decided to supplement recruitment with convenience sampling (Table 2). At the CAB’s suggestion, we hired a part-time local businesswoman to assist with recruitment through her small franchise (selling snow cones) that served diverse community members. She was trained in the talking points and provided flyers about the project. After an initial contact, the recruiter then provided data about potential recruits for the research team to evaluate for participation.

Stakeholder meetings

Within each county, one sampling method was used to recruit three groups of stakeholders to attend a series of three meetings each lasting one to one and a half hours (total of 18 meetings with 9 in each county). All meetings were conducted in Spanish and English and held at convenient times and locations within the same 6 month period. Team members assisted persons with low literacy to understand and participate in activities. The first session provided an orientation about chronic pain and study procedures, including a video of a Hispanic patient describing her experience with chronic pain and a Hispanic primary care physician discussing his approach to managing chronic pain and challenges. The second session was led by an expert facilitator, one per community, and addressed a focused question: “What services or programs are needed to improve the lives of persons with chronic pain?” The participants generated ideas in response to this focused question in a brainstorming meeting structured by the nominal group technique. Developed by social-psychologists, the nominal group technique is the most commonly used structured group method to generate, combine, and prioritize ideas [34]. Initially, participants separately respond to a focused question and then list their ideas “round robin” style. All ideas are reviewed by the group in a facilitated discussion, categorized, and rated. This approach generates diverse ideas within a short timeframe, allowing each individual to contribute instead of only the most outspoken [19]. In the last session, participants grouped ideas into categories and rated separate ideas on both importance and feasibility on five point Likert-type scales. Study protocols were reviewed by the University of Texas Health Science Center at San Antonio Institutional Review Board and determined to be non-regulated research or exempt.

Analysis

Results were analyzed using qualitative and quantitative methods. To examine differences in demographic characteristics of participating stakeholders in the two counties (e.g., demographic differences between two sampling methods), the chi-square test or Fisher’s Exact test was used for categorical variables and the two independent sample t test with unequal variances assumption was used for continuous variables. Because each group of stakeholders sorted their brainstormed ideas into somewhat different groups, three members of the research team independently reviewed and developed categories for the ideas generated by participating stakeholders [35]. Final categories for coding were developed after a discussion of differences among the categories that were previously generated. A final coding of all ideas was conducted by two coders and differences resolved after review by the research team.

A database was created of all community stakeholders’ rankings of their group’s ideas in regard to importance to improve outcomes of persons with chronic pain and feasibility of implementation on a five-point Likert-type scale (i.e., not at all important, somewhat important, very important, extremely important; feasibility scale constructed similarly). For each participant, the mean of their ratings on importance of all ideas within a unique category was calculated and another mean calculated for ratings on feasibility of all ideas within a category. Then these participant-specific mean ratings for each category on each dimension were averaged for all participants within the same community. The mean rating for each category on each of the two dimensions was compared between the two communities using two-sample t test with unequal variances assumption. Lastly, the CAB in each community reviewed these results and presented them to community leaders in order to develop strategies and research projects addressing the highest priorities.

Results

Characteristics of the two groups of stakeholders recruited using two different sampling methods show many similarities (Table 2). The participants recruited by snowball sampling and purposive-convenience sampling were: mean age 58 versus 57 years, 69 versus 65 % women, and 84 versus 89 % preferring English (all p > .05). The distribution of occupations also did not differ (p = 0.47). However, the snowball sampling strategy had a larger proportion of Hispanic participants than purposive-convenience sampling (87 versus 73 %, respectively, p = 0.049) and a larger proportion of participants with a disability (47 versus 10 %, p = <0.001).

Using the snowball recruitment strategy, 67 potential participants were contacted, 65 (97 %) were eligible, and 55 (85 %) consented to participate. Of these 55 stakeholders, 52 (95 %) attended the orientation meeting and 36 (65 %) attended all meetings (Fig. 1). In the other community, purposive sampling recruited 21 eligible stakeholders but was supplemented by convenience sampling to increase timely recruitment. Using this combined sampling method, 71 stakeholders were contacted of whom 69 (97 %) were eligible, 62 (90 %) consented, and 36 (58 %) attended. Overall, 26 (42 %) individuals participated in all meetings (Fig. 1). As shown in Table 3, within each community, the same recruitment strategy recruited three separate stakeholder groups who attended the series of three meetings but snowball sampling consistently yielded higher attendance rates to all meetings. Comparison of characteristics of stakeholders who attended the orientation session (Table 4 left columns) reveals that in the first meeting, snowball sampling resulted in a higher proportion of Hispanic participants than purposive-convenience sampling (90 vs. 75 %, respectively p = 0.052) as well as persons with disabilities (48 vs. 6 %, p < 0.001), respectively. Among participants attending all three sessions (Table 4 right columns), the significant difference in disability status persisted (p = 0.006).

Table 3 Stakeholder participation by sampling method groupa
Table 4 Characteristics of stakeholders within each sampling method group attending first orientation session and three sessions

After brainstorming ideas about interventions needed to improve outcomes of community members with chronic pain (Sessions 1–2), stakeholders rated the priority of each idea on a five-point Likert type scale regarding importance and feasibility to implement. The ideas that were generated by each community and their importance rating are included (Additional file 1). Overall, importance ratings were higher than feasibility ratings (Table 5). For six of the eight categories, ratings on importance did not differ significantly between stakeholders recruited by the two sampling methods. Both stakeholder groups rated professional chronic pain support as very important on average. Notably, specific ideas categorized under professional services and support included a variety of non-pharmacologic sources of care such as a physical therapist, nurse counseling, and pain management support regarding mental health and other complications (Additional file 1). Stakeholders recruited by purposive/convenience sampling rated massage therapy significantly higher (diff = −0.35, p = 0.045) while stakeholders recruited by snowball sampling rated nutritional programs and city improvements/transportation services for persons with chronic pain more highly but the difference was significant only for the latter (diff = 0.62, p = 0.004).

Table 5 Importance and feasibility of needed pain management services and support from community stakeholders grouped by recruitment methoda

None of the ratings on feasibility of implementing these interventions differed significantly between the groups of stakeholders, with most categories rated as being feasible or very feasible. The largest difference in feasibility ratings between the groups was observed for nutritional programs, which was rated as being more feasible by the stakeholders recruited by snowball sampling (diff = 0.50, p = 0.059).

Discussion

In a seminal report, Unequal Treatment, the Institute of Medicine highlighted limited acceptance and involvement in research by community members as a major barrier to sustainable implementation and adoption of health care advances [36, 37]. Engaging community members starting from earliest stages of developing a research project can increase both acceptance and potential sustainability of research results within the community [10, 38]. Community partnerships are especially critical in promoting participation from hard-to-reach populations in all phases of research [3941]. However, methods to elicit community priorities, especially those of vulnerable, hard-to-reach communities, have been subject to limited evaluation. This study offers valuable insights from implementing two common non-probability sampling methods to recruit individuals from similar rural, predominantly Hispanic communities.

A major finding from this project was that purposive sampling, intended to achieve participation from diverse community constituencies, was challenging to implement largely because of limited cooperation from employers, especially those based outside of the community. In their review of sampling methods to engage stakeholders to identify research priorities, O’Haire and colleagues noted that databases of potential participants are often used for purposive recruitment [10]. For example, purposive sampling has been successfully employed to recruit physicians and leaders within a community [42]; however, at the population level, it is highly unlikely that a purposive sampling framework would be available and, if not, it would require significant effort to build. Because our research team lacked a database for purposive sampling and recruitment was flagging, the CAB in community 2 recommended that we transition to a convenience sampling method, as in other community engagement studies [43]. Convenience sampling was accomplished by hiring a part-time small business owner who interacted with a diverse cross-section of community members on a daily basis. She was trained by the CAB and our research team in recruitment methods and, in a short time, was able to identify potentially eligible low income, Hispanic residents with or affected by chronic pain as caregivers. A pragmatic approach to sampling that combines methods to accommodate challenges such as nonparticipation and inability to locate a target population has been adopted or promoted by others to engage hard-to-reach populations [4446].

Another key finding was that snowball sampling recruited a larger number of eligible stakeholders. Other studies have found that snowball sampling is particularly effective in hard-to-reach or ‘hidden’ populations because it takes advantage of established social networks of persons with characteristics of interest [4749]. The CAB in community 1 also served a vital role in operationalizing snowball sampling by identifying the “seeds” who were individuals with or affected by chronic non-cancer pain. This initial recruitment of seeds by the CAB can be regarded as a form of convenience sampling – which in this case resulted in a higher proportion of persons with disabilities with pain, possibly because it was evident to CAB members that these community members suffered from chronic pain. Thus, our snowball sampling method actually integrates initial convenience sampling.

In addition, attendance to all meetings was higher for the snowball sampling recruits than for purposive plus convenience sampling. Promotion of our project by the “seed,” who is a known community member, may have encouraged attendance. However, snowball sampling also recruited more participants who were more severely affected by chronic pain, as manifested by being disabled, compared with those recruited using purposive plus convenience sampling. Thus, careful attention to the characteristics of the seeds and their contacts is needed to promote a balanced representation of stakeholders in the community. Gratifyingly, both sampling methods resulted in recruitment of low-income, predominantly Hispanic community members though snowball sampling recruited a higher proportion of Hispanics.

Despite recruitment with different sampling methods, stakeholders affected by chronic pain in both counties generated relatively similar ideas and priorities for services and support needed to improve outcomes of persons with chronic pain. Professional treatment for chronic pain was rated as very important but this category includes multiple types of professionals delivering non-drug therapies. All community members also gave a high rating to massage therapy but the snowball sampling group in community 1 judged this to be less important, possibly reflecting this group being more disabled. Small studies suggest that the impact of massage on pain or functional outcomes may be diminished for disabled persons [50]. Other priorities included: non-professional support for chronic pain such as group meetings and activities such as arts and crafts; community education about chronic pain; and water therapy. These priorities differ significantly from those of persons with chronic pain who have been treated with long-term opioid analgesics because the latter group focuses primarily on logistics and challenges of obtaining these drugs [51, 52]. These data also suggest that residents of these communities may have a better appreciation of the multi-faceted nature of pain management, in alignment with new national guidelines to use non-opioid drugs and complementary therapies as first line approaches to manage chronic pain [53]. A fertile line of research follows from these community priorities to study implementation and outcomes of evidence-based non-pharmacologic therapy in under-resourced, low income populations.

Limitations of this study include its location – rural Texas counties – that may not be relevant to other low income, predominantly minority communities. However, in support of generalizability to rural Hispanic communities, these two groups independently arrived at relatively similar ideas and priorities regarding services for chronic pain. The study had a relatively small number of participants but our sample size is similar to other stakeholder engagement studies [9]. The robustness of our results may have been increased because we convened three small groups within each county, fostering generation of more ideas through greater opportunities for each stakeholder to contribute. Another limitation relates to the adaptive design of our stakeholder engagement strategy. We transitioned from purposive to convenience sampling to achieve our recruitment goals. However, the similarity of priorities elicited from the two distinct communities supports the value of this combined approach to engage a hard-to-reach population.

The structure of our community partnership also likely contributed to the success of this engagement activity. We had a highly respected lead community partner in each community who spearheaded the establishment of a CAB to guide our stakeholder engagement activities. These CABs now serve as the foundation for ongoing community initiatives to operationalize and conduct research based on topics identified by the stakeholders.

Implications for Community Engagement- Implementation of sampling methods to recruit participants for community engaged research need to be guided by community partners. For our study, we relied heavily on two CABs to operationalize different recruitment methods in two similar rural, predominantly Hispanic communities. We found that purposive sampling was challenging to implement due to employers’ lack of cooperation, preventing access to diverse community constituencies; therefore, the CAB directed the research team to transition to convenience sampling. The snowball sampling method was more straightforward to implement and resulted in larger numbers of participants both initially and throughout the series of meetings. Furthermore, it yielded a higher proportion of Hispanic participants whose viewpoints were especially important to solicit in these majority Hispanic communities.

Conclusions

This study informs methods to engage stakeholders from vulnerable communities to identify research priorities by finding that snowball sampling conducted in partnership with a community advisory board achieved higher attendance rates and greater representation from low income Hispanics. Purposive sampling was more difficult to implement and required guidance from the community advisory board to augment recruitment with a convenience sampling approach. Nevertheless, stakeholders from both communities developed similar research priorities, focusing on diverse non-pharmacologic approaches that are not available in the community to address chronic pain. Future studies need to build on this novel study by examining associated resources and costs for differing sampling strategies to be utilized in hard-to-reach communities that need to be prioritized for research initiatives.

Abbreviations

CAB:

Community advisory board

PCORI:

Patient-Centered Outcomes Research Institute

References

  1. Woolf SH, Zimmerman E, Haley A, Krist AH. Authentic engagement of patients and communities can transform research, practice, and policy. Health Aff (Millwood). 2016;35(4):590–4.

    Article  Google Scholar 

  2. Ford JG, Howerton MW, Lai GY, et al. Barriers to recruiting underrepresented populations to cancer clinical trials: a systematic review. Cancer. 2008;112(2):228–42.

    Article  PubMed  Google Scholar 

  3. Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington: National Academies Press; 2001.

    Google Scholar 

  4. Frank L, Basch E, Selby JV. The PCORI perspective on patient centered outcomes research. JAMA. 2014;312(15):1513–4.

    Article  CAS  PubMed  Google Scholar 

  5. George S, Duran N, Norris K. A systematic review of barriers and facilitators to minority research participation among African Americans, Latinos, Asian Americans, and Pacific Islanders. Am J Public Health. 2014;104(2):e16–31.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Cowan K, Oliver S. The James Lind Alliance guidebook version 6. 2016. http://www.jla.nihr.ac.uk/jla-guidebook/. Accessed 20 Sept 2016

  7. Minkler M. Linking science and policy through community-based participatory research to study and address health disparities. Am J Public Health. 2010;100 Suppl 1:S81–7.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Kwon S, Rideout C, Tseng W, et al. Developing the community empowered research training program: building research capacity for community-initiated and community-driven research. Prog Community Health Partnersh. 2012;6(1):43–52.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Hefland M, Berg A, Flum, D, Gabriel, S, Normand, SL. Draft methodology report: “Our questions, our decisions: standards for patient-centered outcomes research” PCORI Methodology Committee. Available at http://www.pcori.org/assets/MethodologyReport-Comment.pdf. Accessed 26 Feb 2013.

  10. O’Haire C, McPheeters M, Nakamoto E, LaBrant L, Most C, Lee K, et al. Engaging stakeholders to identify and prioritize future research needs. Rockville: Agency for Healthcare Research and Quality; 2011. AHRQ Publication No. 11-EHCo44-EF.

  11. Fowler F. Applied social research methods. 5th ed. Thousand Oaks: SAGE Publications, Inc; 2014.

  12. Gilmore GD, Campbell D. Needs and capacity assessment strategies for health education and health promotion. 3rd ed. Sudbury: Jones & Bartlett Publishers; 2005.

  13. Heckathorn DD. Extensions of respondent-driven sampling: analyzing continuous variables and controlling for differential recruitment. Sociol Methodol. 2007;37(1):151–207.

    Article  Google Scholar 

  14. Semaan S, Heckathorn DD, Des Jarlais DC, Garfein RS. Ethical considerations in surveys employing respondent-driven sampling. Am J Public Health. 2010;100(4):582–3.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Ryan KJ, Brady J, Cooke R, Height D, Jonsen A, King P, et al. The Belmont report: Ethical principles and guidelines for the protection of human subjects of research. Washington: National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research; 1979.

    Google Scholar 

  16. Magnani R, Sabin K, Saidel T, Heckathron D. Review of sampling hard-to-reach and hidden populations for HIV Surveillance. AIDS. 2005;19:S67–72.

    Article  PubMed  Google Scholar 

  17. Staniford LJ, Breckon JD, Copeland RJ, Hutchison A. Key stakeholders' perspectives towards childhood obesity treatment: a qualitative study. J Child Health Care. 2011;15(3):230–44.

    Article  PubMed  Google Scholar 

  18. Delbecq AL, Van de Ven A, Gustafson DH. Group techniques for program planning: a guide to nominal group and Delphi processes. Minneapolis: Scott Foresman Company; 1975.

    Google Scholar 

  19. Gallagher M, Hares T, Spencer J, Bradshaw C, Webb I. The nominal group technique: a research tool for general practice? Fam Pract. 1993;10(1):76–81.

    Article  CAS  PubMed  Google Scholar 

  20. US Census Bureau. QuickFacts Frio County, Texas. http://www.census.gov/quickfacts/table/PST045215/48163. Accessed June 28, 2016.

  21. US Census Bureau. QuickFacts Karnes County, Texas. http://www.census.gov/quickfacts/table/PST045215/48255,48163. Accessed June 28, 2016.

  22. Institute of Medicine. Relieving pain in America: a blueprint for transforming prevention, care, education and research. Washington: National Academies Press; 2011.

    Google Scholar 

  23. Turner BJ, Burg FD. A consumer/medical educator conference: new objectives for the medical curriculum. J Gen Intern Med. 1986;1(5):323–7.

    Article  CAS  PubMed  Google Scholar 

  24. Luborsky MR, Rubinstein RL. Sampling in qualitative research rationale, issues and methods. Res Aging. 1995;17(1):89–113.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Whitmarsh L, Swartling ÅG, Jäger J. Participation of experts and non-experts in a sustainability assessment of mobility. Environ Policy Gov. 2009;19(4):232–50.

    Article  Google Scholar 

  26. Zanini C, Sarzi-Puttini P, Atzeni F, Di Franco M, Rubinelli S. Building bridges between doctors and patients: the design and pilot evaluation of a training session in argumentation for chronic pain experts. BMC Med Educ. 2015;15:89.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Rosas SR, Kane M. Quality and rigor of the concept mapping methodology: a pooled study analysis. Eval Program Plann. 2012;35(2):236–45.

    Article  PubMed  Google Scholar 

  28. Sadler GR, Lee HC, Lim RS, Fullerton J. Recruitment of hard-to-reach population subgroups via adaptations of the snowball sampling strategy. Nurs Health Sci. 2010;12(3):369–74.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Kendall C, Kerr LR, Gondim RC, et al. An empirical comparison of respondent-driven sampling, time location sampling, and snowball sampling for behavioral surveillance in men who have sex with men, Fortaleza, Brazil. AIDS Behav. 2008;12(4 Suppl):S97–104.

    Article  PubMed  Google Scholar 

  30. Carballo-Diéguez A, Balan I, Marone R, et al. Use of respondent driven sampling (RDS) generates a very diverse sample of men who have sex with men (MSM) in Buenos Aires, Argentina. PLoS One. 2011;6(11):e27447.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Pascoe SW, Veitch C, Crossland LJ, et al. Patients' experiences of referral for colorectal cancer. BMC Fam Pract. 2013;14:124.

    Article  PubMed  PubMed Central  Google Scholar 

  32. City-Data.com. Karnes County, Texas (TX). http://www.city-data.com/county/Karnes_County-TX.html. Accessed June 28, 2016.

  33. City-Data.com. Frio County, Texas (TX). http://www.city-data.com/county/Frio_County-TX.html. Accessed June 28, 2016.

  34. Witkin BR, Altschuld JW. Planning and conducting needs assessments: a practical guide. Thousand Oaks: SAGE Publications, Inc; 1995. p. 167–71.

    Google Scholar 

  35. Ulin PR, Robinson PE, Tolley EE. Qualitative methods in public health: a field guide for applied research. San Francisco: Jossey-Bass; 2005.

    Google Scholar 

  36. Institute of Medicine. Unequal treatment: Confronting racial and ethnic disparities in health care. Washington: National Academies Press; 2003.

    Google Scholar 

  37. Coye MJ, Aubry WM, Yu W, Health Technology Center. The “tipping point” and health care innovations: Advancing the adoption of beneficial technologies. Washington: National Committee for Quality Health Care; 2003.

    Google Scholar 

  38. Petruney T, Harlan SV, Lanham M, Robinson ET. Increasing support for contraception as HIV prevention: stakeholder mapping to identify influential individuals and their perceptions. PLoS One. 2010;5(5):e10781.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Johnston LG, Sabin K. Sampling hard-to-reach populations with respondent driven sampling. Methodol Innov. 2010;5(2):38–48.

    Google Scholar 

  40. MacFarlane A, O'Donnell C, Mair F, et al. REsearch into implementation STrategies to support patients of different ORigins and language background in a variety of European primary care settings (RESTORE): study protocol. Implement Sci. 2012;7:111.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Ryan M, Scott DA, Reeves C, et al. Eliciting public preferences for healthcare: A systematic review of techniques. Health Technol Assess. 2001;5(5):1–186.

    Article  CAS  PubMed  Google Scholar 

  42. Firth H, Todd A, Bambra C. Benefits and barriers to the public health pharmacy: A qualitative exploration of providers' and commissioners' perceptions of the Healthy Living Pharmacy framework. Perspect Public Health. 2015;135(5):251–6.

    Article  PubMed  Google Scholar 

  43. Carter-Edwards L, Lowe-Wilson A, Mouw MS, et al. Community member and stakeholder perspectives on a healthy environment initiative in North Carolina. Prev Chronic Dis. 2015;12:E127.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Gibbs L, Waters E, de Silva A, et al. An exploratory trial implementing a community-based child oral health promotion intervention for Australian families from refugee and migrant backgrounds: a protocol paper for Teeth Tales. BMJ Open. 2014;4:e004260.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Teddlie C, Yu F. Mixed methods sampling: A typology with examples. J Mixed Methods Res. 2007;1(1):77–100.

    Article  Google Scholar 

  46. Watters JK, Biernacki P. Targeted sampling: Options for the study of hidden populations. Soc Probl. 1989;36(1):416–30.

    Article  Google Scholar 

  47. Barendregt C, van der Poel A, van de Mheen D. Tracing selection effects in three non-probability samples. Eur Addict Res. 2005;11(3):124–31.

    Article  PubMed  Google Scholar 

  48. Neille J, Penn C. Beyond physical access: a qualitative analysis into the barriers to policy implementation and service provision experienced by persons with disabilities living in a rural context. Rural Remote Health. 2015;15(3):3332.

  49. Hadland SE, DeBeck K, Kerr T, Feng C, Montaner JS, Wood E. Prescription opioid injection and risk of hepatitis C in relation to traditional drugs of misuse in a prospective cohort of street youth. BMJ Open. 2014;4(7):e005419.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Cooke M, Emery H, Brimelow R, Wollin J. The impact of therapeutic massage on adult residents living with complex and high level disabilities: a brief report. Disabil Health J. 2016;9(4):730–4.

    Article  PubMed  Google Scholar 

  51. Simmonds MJ, Finley EP, Vale S, Pugh MJ, Turner BJ. A qualitative study of veterans on long-term opioid analgesics: barriers and facilitators to multimodality pain management. Pain Med. 2015;16(4):726–32.

    Article  PubMed  Google Scholar 

  52. McCrorie C, Closs SJ, House A, et al. Understanding long-term opioid prescribing for non-cancer pain in primary care: a qualitative study. BMC Fam Pract. 2015;16:121.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Dowell D, Haegerich TM, Chou R. CDC Guideline for prescribing opioids for chronic pain--United States, 2016. JAMA. 2016;315(15):1624–45.

    Article  CAS  PubMed  Google Scholar 

  54. Patton MQ. Qualitative research methods. 3rd ed. Thousand Oaks: Sage Publications, Inc; 2002.

  55. United States Department of Labor. Fact sheet: general facts on women and job based health. http://www.dol.gov/ebsa/newsroom/fshlth5.html. 2013. Accessed 23 Feb 2016.

  56. Parsons JA. Mamdani M, Bhattacharyya O, Fortin CM, Melo M, Salmon C, et al. Narcotic analgesic utilization amongst injured workers: using concept mapping to understand current issues from perspectives of physicians and pharmacists. BMC Health Serv Res. 2011;11:280.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Auerswald CL, Greene K, Minnis A, Doherty I, Ellen J, Padian N. Qualitative assessment of venues for purposive sampling of hard-to-reach youth. Sex Transm Dis. 2004;31(2):133–8.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Kay Avant PhD, MSN; Jason Hill MS, MS; Jennifer Potter PhD, MPH and the Frio County and Karnes County Community Advisory Boards (CABS) for their contributions to this study.

Funding

This study was supported by the Patient-Centered Outcomes Research Institute by grant ME-13035729. The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Availability of data and material

Full dataset generated and analyzed during this study from the corresponding author at turner@uthscsa.edu.

Authors’ contributions

BJT is the principal investigator on the grant and was responsible for the planning and conduct of all stages of the research study, and drafting and final version this manuscript. MAV, NR, JL, and MD assisted in the planning and conduct of all stages of the study and MAV drafted the manuscript. MAV, NR and YL contributed to the data analyses and interpretation. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

All study protocols were reviewed by the University of Texas Health Science Center at San Antonio Institutional Review Board and determined to be non-regulated research or exempt. Due to this project being a non-regulated research project, verbal consent was obtained from each participant using a script that explained the purpose of the project and the expectations for his/her role during recruitment conversations which were conducted over the telephone prior to stakeholder engagement sessions. The Institutional Review Board deemed the study of minimal risk as it did not include a non-routine intervention or interaction with a living individual for the primary purpose of obtaining data regarding the effect of the intervention or interaction, nor did the researchers obtain private, identifiable information about living individuals.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Barbara J. Turner.

Additional file

Additional file 1:

Importance Rating of Categories of Ideas to Improve Outcomes of Chronic Pain Generated by 2 Sampling Method Groups. Table containing information presented and ranked at the county level of importance and feasibility of interventions for addressing chronic pain within the two communities sampled. (DOCX 24 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Valerio, M.A., Rodriguez, N., Winkler, P. et al. Comparing two sampling methods to engage hard-to-reach communities in research priority setting. BMC Med Res Methodol 16, 146 (2016). https://doi.org/10.1186/s12874-016-0242-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12874-016-0242-z

Keywords