Skip to main content
  • Research article
  • Open access
  • Published:

Using a social marketing framework to evaluate recruitment of a prospective study of genetic counseling and testing for the deaf community



Recruiting deaf and hard-of-hearing participants, particularly sign language-users, for genetics health service research is challenging due to communication barriers, mistrust toward genetics, and researchers’ unfamiliarity with deaf people. Feelings of social exclusion and lack of social cohesion between researchers and the Deaf community are factors to consider. Social marketing is effective for recruiting hard-to-reach populations because it fosters social inclusion and cohesion by focusing on the targeted audience’s needs. For the deaf population this includes recognizing their cultural and linguistic diversity, their geography, and their systems for information exchange. Here we use concepts and language from social marketing to evaluate our effectiveness to engage a U.S. deaf population in a prospective, longitudinal genetic counseling and testing study.


The study design was interpreted in terms of a social marketing mix of Product, Price, Place, and Promotion. Price addressed linguistic diversity by including a variety of communication technologies and certified interpreters to facilitate communication; Place addressed geography by including community-based participation locations; Promotion addressed information exchange by using multiple recruitment strategies. Regression analyses examined the study design’s effectiveness in recruiting a culturally and linguistically diverse sample.


271 individuals were enrolled, with 66.1% American Sign Language (ASL)-users, 19.9% ASL + English-users, 12.6% English-users. Language was significantly associated with communication technology, participation location, and recruitment. Videophone and interpreters were more likely to be used for communication between ASL-users and researchers while voice telephone and no interpreters were preferred by English-users (Price). ASL-users were more likely to participate in community-based locations while English-users preferred medically-based locations (Place). English-users were more likely to be recruited through mass media (Promotion) while ASL-users were more likely to be recruited through community events and to respond to messaging that emphasized inclusion of a Deaf perspective.


This study design effectively engaged the deaf population, particularly sign language-users. Results suggest that the deaf population’s cultural and linguistic diversity, geography, and forms of information exchange must be taken into account in study designs for successful recruitment. A social marketing approach that incorporates critical social determinants of health provides a novel and important framework for genetics health service research targeting specific, and hard-to-reach, underserved groups.

Peer Review reports


This article evaluates the Deaf Genetics Project’s (DGP) degree of success at motivating a hard-to-reach, underserved population – ranging from culturally Deaf to hard-of-hearing individuals – to participate in a genetic counseling and test study through a social marketing framework that addresses critical social determinants of health: social inclusion and social cohesion. Although recruiting individuals in underserved populations into health service research studies is problematic [13], recruiting deafa participants for health service research is even more challenging due to this population’s cultural and linguistic diversity, communication barriers, literacy level, mistrust, and researchers’ unfamiliarity with deaf people [47]. Researchers in previous studies have addressed these issues in various ways by building relationships with deaf organizations [811], receiving cultural sensitivity or awareness training [8], ensuring that research procedures, e.g., questionnaires or focus groups, are accessible in the participants’ language [811], and more recently, by involving deaf individuals in the research process from study design to data interpretation [12]. However, there are little empirical data on effective strategies to promote deaf individuals’ participation in health service research [13, 14], and no data directly related to participation in genetics health service research. Therefore, the deaf population is a group least addressed in genetics health promotion and health care delivery efforts. However, we anticipate more genetic services devoted to common adult conditions, such as cancer, heart disease and diabetes; and with personalized genomic medicine everyone may benefit from genetic services in their lifetime. Therefore, research on effective methods for motivating deaf individuals to participate in genetics health service research is essential for reducing health disparities.

For many within the U.S. culturally Deaf community, American Sign Language (ASL) is the primary language and English (either spoken or written) is the second language. However, language preference, mode of communication, and identities/cultural affiliation (culturally Deaf, culturally hearing, bicultural) vary within the deaf population [15] due to factors such as experiences interacting with deaf and hearing individuals, amount of residual hearing, upbringing, and school setting [16], such that spoken English is some individuals’ primary language. To a large degree, ASL-users constitute a linguistic and cultural minority group who view their deafness as human variation [17, 18]. As with other linguistic and cultural minorities, institutional, socio-cultural, and educational factors affect sign language-users’ knowledge, acceptance, and use of health-related information and services [4]. Many health researchers face difficulties working with sign language-users due to these cultural and linguistic differences [7, 19]. Moreover, when developing effective strategies to promote genetic counseling and testing, or genomic medicine more generally, researchers must consider often overlooked barriers experienced by sign language-users, such as inadequate science education and few opportunities for incidental learning of genetics terms [4, 20, 21] that create challenges understanding genetics concepts. Importantly, researchers must also consider the Deaf community’s general distrust towards genetics due to perceptions of cultural insensitivity and the history of eugenics [22].

Social marketing is effective for recruiting underserved or hard-to-reach populations into research [3, 23]. In this context, commercial marketing tools facilitate voluntary behavior change for the benefit of individuals, groups, or society rather than commercial gain [24, 25]. Social marketing focuses on the targeted audience’s needs, which for the deaf population includes recognizing their cultural and linguistic diversity, the geography of Deaf communities, and that sign language-users are a collective group [18] who use an alternate networking system for information exchange [26]. Importantly, social marketing techniques increase both social inclusion – the ability for a community, such as the Deaf community, to be included in social systems and relationships [27] -- and social cohesion – the degree to which values, a sense of connectedness, trust and familiarity are shared among a group [28]. Programs that increase these factors improve a community’s well-being [2729].

Here we describe our study design and interpret our effectiveness at motivating deaf individuals, particularly sign language-users, to participate in this research through a social marketing framework that emphasizes social inclusion and social cohesion. To our knowledge, this is the only study to examine outcomes of genetic counseling and testing in the deaf population; hence successful recruitment and retention were essential for generating data to fill the gaps in knowledge. More generally, by interpreting our study’s recruitment through a social marketing perspective, our results suggest that a social marketing approach can provide a novel and important framework for genetics health service research hoping to target specific, and hard-to-reach, underserved groups such as the deaf population in general and especially the Deaf community who use sign language.


Overview of the deaf genetics project

The Deaf Genetics Project was a prospective, longitudinal study to examine the impact of genetic counseling and genetic testing on deaf adults and the Deaf community. The study focused on testing two deaf genes, GJB2 and GJB6 (also called by their protein products, Connexin 26 and Connexin 30, respectively), which for some deaf individuals provides a genetic explanation for why they are deaf. The Deaf Genetics Project was designed by a multi-institutional, multi-disciplinary team of Deaf, hard-of-hearing and hearing investigators. Research staff included audiologists and board-certified genetic counselors. Among the four audiologists, one was a certified sign language interpreter, one was familiar with ASL, and two were not familiar with ASL. None of the four genetic counselors were able to communicate directly with participants using ASL. Thus, three certified ASL/English interpreters also were members of the project staff, and an interpreter was available for all participants during the course of the research protocol unless the participant opted to use spoken English without an interpreter. The goals of the DGP were to enroll 250 deaf individuals in ~2 year period, to determine for each participant if they had GJB2 or GJB6 genetic deafness, and to understand the impact of this genetic information on participants. Individuals who were at least 18 years old with an unexplained sensorineural deafness since an early age (defined as birth to age 6 years) were eligible to participate. Participants were recruited over a period of 25 months from the Los Angeles, Bay Area, and Riverside areas of California.

The study protocol involved three steps: 1) audiology session to confirm sensorineural deafness, 2) pre-test genetic counseling to obtain informed consent for genetic testing, and 3) post-test genetic counseling to disclose genetic test results. For all three stages, individuals selected one of four locations for their participation: University of California Los Angeles (UCLA), California State University Northridge (CSUN), California School for the Deaf-Fremont (CSDF), or California School for the Deaf-Riverside (CSDR). Participants determined eligible to participate in steps 2 and 3 were asked to complete a series of four questionnaires: following audiology, following pre-test genetic counseling, and 1-month and 6-months following disclosure of genetic test results. The questionnaires assessed demographic factors, reasons for genetic testing, attitudes toward genetic testing, knowledge and understanding of genetics and genetic testing, cultural affiliation and deaf identity, and a variety of psychological and behavior measures. Participants were informed that DNA samples would only be used for testing the GJB2 and GJB6 genes. All DNA samples were discarded at the end of the study. This research study was conducted in compliance with the Helsinki Declaration and approved by the institutional review board at the University of California Los Angeles (#10-001193) and by the Committee for Protection of Human Subjects at the California State University-Northridge. Details on the participants and study protocol are depicted in Figure 1 and also have been published elsewhere [3032].

Figure 1
figure 1

Deaf genetics project study protocol. Note: YO = years old.

Overview of the social marketing framework

In this section we describe how the DGP study design can be viewed through a social marketing mix of product, price, place, and promotion [24, 25, 3338] (depicted in Figure 2).

Figure 2
figure 2

Social marketing framework used by deaf genetics project. Note: GTC = genetic testing and counseling; UCLA = University of California Los Angeles; CSUN = California State University Northridge; CSDF = California School for the Deaf-Fremont; CSDR = California School for the Deaf-Riverside.


Following Kotler & Lee [39], we divided product into 3 parts. Core product, which is what people gain when they perform the behavior or the desired benefit from the public health perspective, was to increase genetic counseling and testing awareness in the deaf population. Actual product is the desired behavior or benefit from the potential participant’s perspective. Because conveying the significance of genetic information is key to motivate participation and decision making [40], our actual product was multi-pronged: learn why he/she is deaf, give back to the Deaf community, and educate society-at-large about what genetic information means to them. Augmented product is the tangible objects and services used to facilitate change. By applying a multi-layered cultural accessibility approach to the study design, our augmented product is the improvement in health service for deaf individuals and the Deaf community that occurs when providers are culturally competent and barriers to access are reduced [41]. Our multi-layered cultural accessibility approach was facilitated by our multidisciplinary team composed of Deaf, hard-of-hearing, and hearing researchers who worked to ensure the study design’s appropriateness and to increase the Deaf community’s trust [6].


Price is the cost that the target market associates with adopting the desired behavior [39]. An important consideration for deaf individuals is the psychological cost associated with communication -- within medical settings, with healthcare providers, and with researchers [41]. Importantly, this cost increases social exclusion and decreases social cohesion between participants and researchers. However, our use of multiple communication approaches to embrace a variety of communication modes fostered social inclusion and cohesion and minimized this cost. Videophone, text telephone (TTY), and instant messaging, which are not prevalent in health care settings, as well as voice telephone and email, were available in the DGP research setting for enabling distance contact between project personnel and participants. Research staff included 3 certified ASL/English sign language interpreters who were available for in-person sessions. The interpreters were trained to convey complex genetics terminology and concepts with lay and visual terms in ASL to ensure high quality and consistent interpreting across participants. They were bilingual and bicultural, and their presence during sessions helped ensure clear and appropriate communication between the genetic counselor and the participant [42]. The genetic counselors were trained in cultural sensitivity which enabled Deaf participants to raise questions without having to explain or educate the professionals about their condition and values. Questionnaires were translated into ASL and available in ASL (video streaming), English text, and bilingual format of ASL video + English text.


Place is where and when the target behavior is performed and is an underused social marketing component [43]. However, place can be considered an essential component of our enhancement of social inclusion because in our study design we recognized that sign language-users prefer to reside near educational opportunities for deaf individuals, creating Deaf communities where residential schools or mainstreaming programs are located. Thus, we developed three community-based participation locations (CSDF, CSDR, CSUN), and one medically-based location (UCLA).


The purpose of “Promotion” is to inspire action in target audiences [39]. In a social marketing framework, the DGP’s use of branding, messaging, and recruitment channels can be considered promotion techniques to recruit participants in the following ways.


The name “Deaf Genetics Project” with the image of a DNA helix overlayed with two hands signing DNA (Figure 3) was developed to uniquely identify our study and resonate with many deaf individuals regardless of their attitude toward genetics and genomics health service research. The brand identity provided a look and feel that helped to unify and integrate all study activities. To maximize impact and effectiveness, the “Deaf Genetics Project” brand and logo were used on all study materials, including website, brochures, postcards, and flyers.

Figure 3
figure 3

Deaf genetics project logo.


Messaging strategies are an essential component of social marketing and are a widely used health communication campaign [44]. Our development of two primary messages to inform the deaf population about the study themes, promote study activities, and to inspire participation can be considered messaging. One message focused on an individual’s curiosity to learn why they are deaf. Another message focused on generative fulfillment[23], i.e., sharing their experience could educate society about the impact of genetic testing on the Deaf community. Secondary messages endorsed social cohesion and included that the team was composed of Deaf, hard-of-hearing, and hearing researchers, and that genetic testing was not intended to cure deafness or to affect an individual’s hearing [30]. This messaging was meant to reassure Deaf individuals that our goal was to benefit the Deaf community, not threaten it --- a goal recently reiterated in McKee et al. [7]. Importantly, positive framing was used [45]. Recruitment materials and DGP staff intentionally used terms “deaf” or “hard-of-hearing” instead of “hearing loss” or “hearing impaired;” “chance” instead of “risk;” “variant” instead of “mutation;” and avoided “normal” or “abnormal.”

Recruitment channels

Four recruitment channels were used: mass media, community events, organizations, and personal media. All recruitment strategies encouraged interested individuals to directly contact DGP personnel, or to allow DGP staff to contact them (with their permission) to initiate the study protocol process.

For recruitment through mass media, the DGP developed a website (, multicolored brochure, multicolored visual postcard, and black and white flyer. Because deaf individuals generally prefer a visual-oriented approach, the DGP primarily used a visual discourse model, which fostered social inclusion. Recruitment items included the DGP logo and messaging to disseminate information about the project to the target population and to people involved with the deaf population via internet, mass mailings, listservs, and displays. The website provided information with ASL videos and English text to increase accessibility. Information on the website included DGP’s goals, eligibility criteria, procedures, frequently asked questions, staff, directions, and outreach efforts. English terms were simple and we used written lay terms to ensure literacy accessibility to the deaf population, for whom English is a second language [46]. A link to the website’s URL was included in messages through listservs to reach many people at once [38]. The multicolored brochure included the DGP logo, the purpose of DGP, criteria, estimated outcome, possible benefits and risks of participating, frequently asked questions, contact information, and DGP’s website address. The multicolored visual postcard used more pictures and simpler text messaging than the brochure, and included only the DGP logo, purpose, and website address. The black and white flyer included the study purpose, location, criteria, research procedures, and contact information. A variety of mass media approaches were used to complement the strengths and weaknesses of each approach. As one example, brochures and postcards can convey in-depth information, especially about complex issues, often at low cost, and often promote follow-up by interested individuals to request more information [38]. As another example, flyers are useful for generating awareness using limited amount of information, and can be placed in high-visibility places. Except for the study website, mass media approaches were used only in the UCLA, CSUN, and CSDF geographic locations.

A culturally relevant recruitment channel is community events regularly attended by Deaf individuals, e.g., Deaf Festivals, DeafNation Expos (, or social events sponsored by deaf organizations such as homecoming festivals at deaf residential schools. Community events allowed the DGP to staff an exhibit booth and disseminate information in multiple direct and interactive ways in an informal, brief period using a structured message. As part of the messaging kit, the booth included a banner with DGP logo to draw attention to the booth, a video describing the DGP in ASL, buccal brushes to illustrate DNA collection, and a binder containing information about genetic counseling and testing using visual aids. DGP used Deaf peer communicators at these events to promote study participation. Direct peer communication can be very effective, especially if the person is seen as credible to the target audience [47], questions can be answered immediately, and the message can be personalized to address particular benefits and barriers important to that person of a similar cultural background. This recruitment channel was used in all four participation locations.

The study team also gave structured informational presentations or staffed an exhibit booth at organizations including schools, religious institutions, non-profit organizations e.g., Greater Los Angeles Agency on Deafness and the Hearing Loss Association, and deaf-related conferences e.g., California Association for the Deaf. These venues are a good way to reach professionals in the target group, and DGP’s affiliation with these organizations increased the likelihood that the target audience would pay attention to the message. Moreover, conference sessions can provide information in a memorable way, and materials can be distributed efficiently [38]. This recruitment channel was used in all four study participation locations.

Finally, people are more likely to say yes to a request from someone they know and like [48]. Personal communication can be more effective than other traditional social marketing strategies because of its perceived credibility and impartiality [49]. Through social networks -- “personal media channels” --such as word of mouth from a friend or family member, or through incidental face-to-face contact with a project team member, information about the DGP could happen naturally.

Data collection

During initial eligibility screening, information about the individual’s primary language and preference for a sign language interpreter when interacting with non-ASL-fluent research staff, available methods for contact, preferred participation location, and primary source for learning about the study (recruitment channel) were recorded. The baseline questionnaire, which was completed immediately following the audiology session, assessed demographic characteristics (age, gender, race/ethnicity, education, income, cultural affiliation) and reasons for genetic testing. Age was dichotomized based on median age of the sample, education was dichotomized as < or ≥4-year college degree, and cultural affiliation was classified as Deaf community, hearing community, both communities, and neither community. Language was categorized as ASL, ASL + English, English, and other (e.g., signed English). As for the group of English-users, they are individuals in this study sample who opted to use the spoken modality with study staff. Methods for contact (price) were coded and dichotomized as sign language-user inclusive (email, videophone, TTY, text only cell phone, pager) or not. Participation location (place) was dichotomized as community-based (CSUN, CSDF, CSDR) or medically-based (UCLA). Participants’ responses to how they learned about the study (promotion) were classified as mass media, community events, organizations, or personal contact. Three potential reasons for genetic testing (product; promotion) are examined (i) ‘to learn why I am deaf/hard-of-hearing’ (a measure of the curiosity messaging); ‘to help research’ (a measure of the generative fulfillment messaging); and ‘to strengthen the Deaf community, ’ (a measure of inclusion of Deaf perspective messaging). For each of these items, respondents rated how strongly they agreed that this was an important reason for genetic testing using a 5-point Likert scale of strongly agree to strongly disagree.


Analyses were performed in this social marketing framework to examine which aspects of communication (price), geography (place), and forms of information exchange (promotion) are important to the culturally and linguistically diverse population of deaf individuals when considering participation in a genetic counseling and testing study. Bivariate associations were assessed using Fisher’s exact test, t-test, or ANOVA. Logistic regression analyses were performed to examine the influence of language on (a) preferred modes of communication between participants and researchers, (b) preference for participation location, and (c) recruitment source. Odds ratios with 95% confidence intervals were also calculated. Linear regression analyses were performed to examine whether the primary and secondary messaging (product; promotion) were associated with participants’ reasons for their interest in genetic testing. Age, gender, education, and recruitment channel were included as covariates as appropriate. Cultural affiliation was not evaluated as a covariate in these analyses because this demographic variable is very strongly associated with language in this sample [30, 31]. Participants whose language was described as “other” were excluded from analyses that included language as a variable due to their small sample size and heterogeneous nature. Logistic regression analyses were conducted using the proc LOGISTIC procedure and linear regression analyses were conducted using the proc MIXED procedure in SAS version 9.2 [50]. Statistical significance was defined as p ≤ 0.05.



During a 27 month enrollment period 271 individuals completed an audiology session and 97% were determined eligible for genetic counseling and testing. Among these 271 individuals, 66.1% preferred to use ASL, 19.9% preferred to use ASL + English, 12.6% preferred to use English, and 1.5% used Pidgin Sign Language, Signed English, or another coded language form (other). Sample characteristics of those completing an audiology session are provided in Table 1.

Table 1 Sample characteristics (N = 271)

Participation rate among those eligible to continue onto the genetic counseling and testing phases was very high (Figure 1): 94.3% of those determined eligible attended the pre-test genetic counseling session, 99.6% provided a DNA sample for genetic testing, and 97.6% of those individuals returned to learn their genetic test results. Questionnaire response rates also were high: 98.9% at baseline, 98.8% following pre-test genetic counseling session, 76.8% 1-month following test result disclosure session, and 75.9% 6-months following test result disclosure session. Of note, those who completed the 6-months questionnaire did not differ significantly from who did not complete it in terms of participant language, recruitment channel, or participation location (all p’s > 0.05).

Price: communication

Linguistic diversity was present in the sample, although most participants preferred to use ASL or ASL + English (Table 1). All participants whose language included ASL, even those who also used spoken English, preferred to interact directly with any non-ASL-fluent research staff via a sign language interpreter, making a sign language interpreter a necessary component of lowering the price of participation. No participant requested a strictly written or lip-reading interaction with research staff. Participants typically used >1 technology for distance communication (Table 2), and technologies varied with preferred language. As shown in Table 3, ASL- and ASL + English-users were more likely to have a videophone (OR = 9.83, 95% CI: 3.26, 29.67; OR = 10.16, 95% CI: 3.06, 33.80, respectively) or a pager (OR = 19.12, 95% CI: 5.42, 67.40; OR = 12.91, 95% CI: 3.42, 48.79, respectively) compared to English-users. In contrast, ASL- and ASL + English-users were less likely to use a voice telephone compared to English-users (OR = 0.12, 95% CI: 0.05, 0.28; OR = 0.09, 95% CI: 0.03, 0.26, respectively). We then classified communication technologies as either sign language inclusive (videophone, pager, TTY, text only cellphone) or not, and found that ASL- and ASL + English-users were more likely to have sign language inclusive technologies for communicating with research staff than English-users (OR = 35.18, 95% CI: 11.96, 103.42; OR = 21.96, 95% CI: 6.37, 75.65, respectively).

Table 2 Distribution of communication technologies, participation locations, and recruitment channels by participant language
Table 3 Logistic regression analyses: effect of participant language on communication technologies, participation location, and recruitment channel

Place: participation locations

The participation sites varied in total number of months of operation with UCLA and CSUN fully operational for 27 months, CSDF for 15 months, and CSDR for 8 months. On average, 2.3 – 4.2 individuals were enrolled per month of operation at each location. Monthly rate of enrollment did not differ significantly across the four locations (F(3,77) = 1.92, p = 0.13). Table 2 gives the distribution of participation locations by preferred language.

We investigated the effect of language on choice of participation location. As shown in Table 3, ASL- and ASL + English-users were more likely to enroll in a community-based location (OR = 2.72, 95% CI: 1.20, 6.14; OR = 3.39, 95% CI: 1.31, 8.74, respectively) compared to English-users who preferred the medically-based location. Specifically, 62.7% (146/233) of participants who preferred to use ASL or ASL + English enrolled at the community-based locations; in contrast, 64.7% (22/34) of English-users enrolled at the medically-based location. A similar trend was noted when we restricted our analysis to enrollment at the community-based location (CSUN) and medically-based location (UCLA) sited in the same geographic area, although it failed to achieve statistical significance (ASL-users OR = 1.49, 95% CI: 0.53, 4.17; ASL + English-users OR = 3.01, 95% CI: 0.95, 9.51).

Promotion: recruitment channels

We implemented 49 distinct recruitment activities during the 25 month recruitment period. Over half (55.1%) of recruitment activities focused on staffing an exhibit booth or giving presentations at events/meetings/conferences sponsored by organizations, 24.5% on staffing an exhibit booth or giving presentations at community events such as DeafNation Expos, and 20.4% on the use of mass media channels. Although only a quarter of recruitment activities focused on community events, this was the most frequently reported channel (41.2%) for learning about the study. From a social marketing perspective, this result indicates that researchers should increase emphasis on community events in any future genetic studies in the Deaf community. Table 2 gives the distribution of recruitment channels by preferred language.

We investigated the effect of language on recruitment channel (mass media, community events, organizations, personal media). For these analyses we focus on those who participated at UCLA, CSUN, and CSDF because all four recruitment channels were used in the geographic areas of these participation locations. As shown in Table 3, ASL-users and ASL + English-users were more likely to be recruited through community events than English-users (OR = 23.18, 95% CI: 3.60, 976.77; OR = 19.63, 95% CI: 2.69, 879.57, respectively). In fact, 47.45% (65/137) of ASL-users and 43.9% (18/41) of ASL + English-users were recruited through community events compared to 3.7% (1/27) of English-users. In contrast, ASL- and ASL + English-users were less likely than English-users to be recruited through mass media (OR = 0.13, 95% CI: 0.04, 0.48; OR = 0.22, 95% CI: 0.05, 1.04, respectively), with 5.11% (7/137) of ASL-users and 7.32% (3/41) of ASL + English-users compared to 22.22% (6/27) of English-users. Language was not significantly associated with recruitment through organizational channels or personal media (Table 3) and roughly half of the participants were recruited through these two channels (Table 2).

Promotion: messaging

We found that 91.7% strongly agreed/agreed that they were interested to learn why they are deaf, and 91.6% strongly agreed/agreed that they were interested in helping research, suggesting that their reasons for genetic testing reflected the curiosity and generative fulfillment primary recruitment messaging. Linear regression analyses demonstrated that the primary messaging resonated with all language groups and through all recruitment channels since neither of these variables were significant predictors of participants’ responses to these genetic testing reasons (p’s > 0.05). We explored whether participants’ genetic testing reasons reflected the secondary messaging that highlighted inclusion of the Deaf perspective in the research process and found that 75.2% of participants strongly agreed/agreed that they were interested in genetic testing to strengthen the Deaf community. Response to this item was significantly associated with language (p = 0.006) and recruitment channel (p = 0.005). Specifically, ASL- and ASL + English-users indicated stronger agreement compared to English-users; and those who learned of the study through community events, organizations, and personal media indicated stronger agreement compared to those who learned of the study through mass media.


Recruiting deaf individuals to participate in health services research has been challenging. Studies suggest that a variety of social, cultural, and linguistic factors that have resulted in the exclusion of deaf individuals from the bulk of health services research must be addressed to overcome recruitment barriers [7]. This article demonstrates the value of explicitly addressing social exclusion and social cohesion in a health service research design to attract the deaf population’s participation, and ultimately to reduce health disparities. Moreover, we demonstrate that these factors can be addressed within a social marketing framework, and that this framework helps assess whether recruitment of an underserved, hard-to-reach population is successful.

Recruitment in the DGP was quite successful. Not only did we achieve our target sample size of 250, but 97% of individuals who attended an audiology session were eligible for the genetic counseling and testing sessions. This result demonstrates that our recruitment and initial eligibility screens effectively targeted the desired study population. Moreover, participant retention was very high, where >90% of eligible participants submitted a DNA sample for genetic testing during pre-test genetic counseling and returned to learn their genetic test results, and 76% of participants responded to the final 6 months post results questionnaire. Importantly, participants retained out to 6-months post-test result disclosure did not differ significantly from those not retained across recruitment channel, participant language, or participation location. These results not only demonstrate that our study design was flexible enough that those recruited felt committed to their participation, but that this study design did not produce a “missingness” bias into the questionnaire data. Given reports that few deaf individuals, particularly sign language-users are seen in genetics clinics [51], these results are highly encouraging. By evaluating recruitment in a social marketing framework, we can specifically determine what aspects of our study design are effective for conducting genetics health service research in the deaf population and the Deaf community and determine how to generalize these aspects to health service research in underserved minority groups.

We utilized a social marketing framework to identify the components of the social marketing mix used in our study design. From this analysis we interpret our study design as reducing the participation “price” by reducing language barriers and attending to the cultural and linguistic differences between researchers and participants that contribute to social exclusion and lack of social cohesion, i.e., by employing a variety of communication technologies, certified sign language interpreters, and culturally sensitive genetic counselors, and by providing survey questionnaires in ASL. Consistent with other studies, e.g., [8, 9, 11], our results demonstrate that ensuring communication between deaf individuals and researchers is an effective means for engaging them in a research project. Furthermore, our findings demonstrate the importance of attending to the linguistic diversity of the deaf population as well as language concordance/discordance between researchers and participants. Videophone and sign language interpreters were used for communication between ASL- and ASL + English-users and non-ASL fluent study personnel while voice telephone and no interpreters were preferred by deaf English-users for interactions with English-using project personnel. Ironically, we faced initial institutional reluctance in the medically-based location to allow videophone use, an internet-based communications device analogous to videoconferencing, due to local interpretation of the U.S. Health Insurance Portability and Accountability Act (HIPAA) regulations which protects the privacy of individually identifiable health information. Videophones are an effective way to remove communication barriers [7] but they require open and non-firewalled connection, which raised the potential for an IP security issue. Although a recent review did not reveal a negative effect of HIPAA on minority health research [1], our experience reveals a lack of cultural awareness in medical institutions of the importance of videophone as a communication device for ASL-users, which could negatively affect health service research in this population. Concerns about violating HIPAA should be eliminated if the deaf person gives their approval to go ahead after appropriate consent.

We can also interpret our results as addressing social inclusion and social cohesion through the “place” element of social marketing. We engaged in statewide recruitment where there are deaf residential schools and deaf organizations, and thus provided three community-based participation locations along with a medically-based location. Previous health-related studies have used community partnerships [10, 11], and we found that developing these partnerships was highly effective for engaging sign language-users with this research. Moreover, because we analyzed participation location by preferred language, our results provide even stronger evidence for the importance of “place”. Not only were ASL- and ASL + English-users more likely to be enrolled at the community-based locations than English-users, but nearly two-thirds of ASL- and ASL + English-users in our sample opted to participate through a community-based location. Hence, using community-familiar locations where sign language-users are concentrated enhanced desirability to participate in health service research. Furthermore, when controlling for geographic location, a similar non-significant trend was observed. Lack of statistical significance may reflect lack of power with the reduced sample size or that efforts to create a culturally and linguistically appropriate environment for ASL-users at the medically-based location were effective to engage sign language-users. Together, these results suggest that culturally and linguistically appropriate environments - which typically are part of the fabric of community-based locations - can be achieved in other locations and effectively engage sign language-users in health service research. Thus, endeavors at developing culturally and linguistically appropriate genetics health services and health service research for the ASL-using community should use a two-pronged approach by focusing efforts on locations with a heavy concentration of ASL-users, such as nearby deaf residential schools or Deaf community service organizations, and creating culturally and linguistically appropriate environments in medically-based settings.

In terms of “promotion,” participants were recruited from all four channels. However, deaf English-users were more likely to learn about the study through mass media, while community events were the most effective strategy for recruiting sign language-users. This result also demonstrates the importance of attending to the diversity of the deaf population. Community events are a common site of recruitment for studies involving Deaf individuals [8, 11, 12, 52, 53] and our study suggests that its effectiveness exceeds other recruitment channels. The importance of community events for recruiting sign language-users is intriguing because a recent review of recruitment strategies for enrolling cultural and linguistic minorities into health service research found that although outreach through community events was a frequent strategy, it was the least successful approach in 77% of 42 studies that employed it [3]. We hypothesize several reasons for our success with community events. First, the Deaf community’s collective nature and desire to socialize with other ASL-users [54] facilitate both the construction of, and attendance at, Deaf-related community events. Secondly, community events offered opportunities for flexible, interpersonal connections between project personnel and potential participants, involved peer communicators (ASL-users) which increased credibility and trust, and demonstrated our commitment to inclusivity by showing collaboration between Deaf and hearing researchers. Furthermore, attending community events demonstrated that research personnel were willing to meet Deaf people in their preferred environments in order to develop trust and rapport. Anecdotally, individuals approaching our exhibit booth at community events frequently had questions regarding DNA sample ownership and this recruitment venue allowed us to explain that the sample would be discarded at the study’s conclusion. It is possible that this practice also served to increase trust and build rapport and thereby increased a sense of social inclusion because community members believed that the program focused on their unique needs [7, 27]. Our experience also demonstrates that deaf English-users are a challenging group to reach directly and more research is needed to identify successful recruitment strategies for this group.

We interpret DGP’s core product as increasing awareness about genetic counseling and testing and the actual product to determine if genetics explains why a person is deaf (curiosity) and to provide an opportunity to educate society-at-large about what genetic testing means to deaf individuals and the Deaf community (generative fulfillment), i.e., giving this group a chance to express their views to the general population through this research. Those were primary recruitment messages and the majority of participants, regardless of preferred language or recruitment channel, endorsed those ideas as their participation reasons. Thus, curiosity and generative fulfillment messaging were successful messaging techniques. Generative fulfillment has been successful in other social marketing campaigns [23], and these findings extend this technique’s effectiveness to recruiting deaf individuals into health service research. Results also suggest that our secondary messaging indicating inclusion of the Deaf perspective would be an effective technique for specifically engaging ASL-users and that this messaging technique would be most effective via community events, organizations, and personal media.

This study has several limitations. Only one recruitment channel was recorded for most participants, although participants might have learned of the DGP through multiple venues. Thus, although it appears that our mass media campaign was least effective for recruiting participants, this channel might have increased effectiveness of the other recruitment channels. We also could not record data on individuals who screened ineligible, thus we cannot comment on each recruitment channel’s relative effectiveness to target eligible participants. Furthermore, given the use of the internet, email listservs, conference presentations, and exhibit booths, it was not possible to track how many individuals were reached through our recruitment methods and channels. Thus, another limitation of this study is that we are unable to comment on the response rate produced by our recruitment approach. In addition, the individuals who chose to participate in the DGP may differ in important ways from those who did not participate. As one example, our study participants were more highly educated and had a higher median income than a national sample of prelingually deaf adults [55]. As another example, deaf individuals attending community events (our major recruitment source) may not be typical of the larger deaf population. Hence, generalizing our findings to the general deaf population should be done with caution.


This paper interprets a study design through a social marketing framework and provides a guide of how to conceptualize, implement and assess a social marketing approach to genetics health services research for the deaf population. Using the social marketing framework allowed us to determine that our study effectively engaged deaf individuals, particularly sign language-users, and suggests that all genetics health service research designs should address social inclusion and cohesion. Results also suggest that the deaf population’s cultural and linguistic diversity, geography, and forms of information exchange must be taken into account in study designs for successful recruitment. A social marketing approach that incorporates critical social determinants of health provides a novel and important framework for genetics health service research targeting specific, and hard-to-reach, underserved groups.


aThe general consensus of the wording usage for various members of our study population is somewhat in flux. Hence for this study, the usage of ‘deaf’ lexicon with lowercase ‘d’ is applied throughout this article to reflect this study population in a broader sense. The term deaf applies to individuals ranging from culturally Deaf, deaf, hard-or-hearing either using sign language or using residual hearing by speech [21]. The usage of ‘Deaf’ with uppercase ‘D’ as widely adopted from Woodward’s convention [56] is used here to exemplify a targeted group of our study who use sign language and identify themselves as a part of Deaf community.



American Sign Language


Analysis of variance


Confidence interval


California School for the Deaf – Fremont


California School for the Deaf – Riverside


California State University Northridge


Deaf Genetics Project


Deoxyribonucleic acid


Health Insurance Portability and Accountability Act


Odds ratio


Text telephone


University of California Los Angeles.


  1. Paskett ED, Reeves KW, McLaughlin JM, Katz ML, McAlearney AS, Ruffin MT, Halbert CH, Merete C, Davis F, Gehlert S: Recruitment of minority and underserved populations in the United States: the Centers for Population Health and Health Disparities experience. Contemp Clin Trials. 2008, 29: 847-861. 10.1016/j.cct.2008.07.006.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Preloran HM, Browner CH, Lieber E: Strategies for motivating Latino couples’ participation in qualitative health research and their effects on sample construction. Am J Public Health. 2001, 91: 1832-1841. 10.2105/AJPH.91.11.1832.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. UyBico SJ, Pavel S, Gross CP: Recruiting vulnerable populations into research: a systematic review of recruitment interventions. J Gen Intern Med. 2007, 22: 852-863. 10.1007/s11606-007-0126-3.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Harmer LM: Health care delivery and deaf people: practice, problems, and recommendations for change. J Deaf Stud Deaf Edu. 1999, 4: 73-110. 10.1093/deafed/4.2.73.

    Article  Google Scholar 

  5. Working with Deaf People: A Handbook for Healthcare Professionals. Edited by: Middleton A. 2010, Cambridge: Cambridge University Press

    Google Scholar 

  6. Meador HE, Zazove P: Health care interactions with deaf culture. J Am Board Fam Pract. 2005, 18: 218-222. 10.3122/jabfm.18.3.218.

    Article  PubMed  Google Scholar 

  7. McKee M, Thew D, Starr M, Kushalnagar P, Reid JT, Graybill P, Velasquez J, Pearson T: Engaging the deaf American Sign Language community: lessons from a community-based participatory research center. Prog Community Health Partnersh. 2012, 6 (3): 321-329. 10.1353/cpr.2012.0037.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Patel JV, Gill PS, Chackathayil J, Ojukwu H, Stemman P, Sheldon L, Meelu S, Lane DA, Tracey I, Lip GY, et al: Short-term effects of screening for cardiovascular risk in the deaf community: a pilot study. Cardiol Res Pract. 2011, 2011: 493546-

    CAS  PubMed  PubMed Central  Google Scholar 

  9. McKee M, Schlehofer D, Cuculick J, Starr M, Smith S, Chin NP: Perceptions of cardiovascular health in an underserved community of deaf adults using American Sign Language. Disability Health J. 2011, 4: 192-197. 10.1016/j.dhjo.2011.04.001.

    Article  Google Scholar 

  10. Zazove P, Meador HE, Reed BD, Sen A, Gorenflo D: Effectiveness of videos improving cancer prevention knowledge in people with profound hearing loss. J Cancer Educ. 2012, 27: 327-337. 10.1007/s13187-011-0292-1.

    Article  PubMed  Google Scholar 

  11. Steinberg AG, Barnett S, Meador HE, Wiggins EA, Zazove P: Health care system accessibility: experiences and perceptions of deaf people. J Gen Intern Med. 2006, 21 (3): 260-266. 10.1111/j.1525-1497.2006.00340.x.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Barnett S, Klein JD, Pollard RQJ, Samar V, Schlehofer D, Starr M, Sutter E, Yang H, Pearson TA: Community participatory research with deaf sign language users to identity health inequities. Am J Pub Health. 2011, 101 (12): 2235-2238. 10.2105/AJPH.2011.300247.

    Article  Google Scholar 

  13. Pollard RQ, Dean RK, O’Hearn A, Haynes SL: Adapting health education material for deaf audiences. Rehabil Psychol. 2009, 54: 232-238.

    Article  PubMed  Google Scholar 

  14. Sadler GR, Lee HC, Lim RSH, Fullerton J: Recruitment of hard-to-reach population subgroups via adaptations of the snowball sampling strategy. Nurs Health Sci. 2010, 12: 369-374. 10.1111/j.1442-2018.2010.00541.x.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Leigh I: A Lens on Deaf Identities. 2009, New York: Oxford University Press

    Book  Google Scholar 

  16. Padden C, Humphries T: Deaf in America: Voices from a Culture. 1988, Cambridge, MA: Harvard University Press

    Google Scholar 

  17. Lane H: Ethnicity, ethics, and the deaf-world. J Deaf Stud Deaf Edu. 2005, 10: 291-310. 10.1093/deafed/eni030.

    Article  Google Scholar 

  18. Ladd P: Understanding Deaf Culture: In Search of Deaf hood. 2003, Tonawanda, NY: Multilingual Matters

    Google Scholar 

  19. Barnett S, McKee M, Smith S, Pearson TA: Deaf sign language users, health inequities, and public health: opportunity for social justice. Prev Chronic Dis. 2011, 8 (2): A45-

    PubMed  PubMed Central  Google Scholar 

  20. Israel J, Cunningham M, Thumann H, Arnos K: Genetic counseling for deaf adults: Communication/language and cultural considerations. J Genet Counsel. 1992, 1: 135-153. 10.1007/BF00962915.

    Article  CAS  Google Scholar 

  21. Hauser P, O’Hearn A, McKee M, Steider A, Thew D: Deaf epistemology: Deafhood and deafness. Am Ann Deaf Winter. 2010, 154 (5): 486-492. 10.1353/aad.0.0120.

    Article  Google Scholar 

  22. Middleton A, Hewison J, Mueller RF: Attitudes of Deaf adults toward genetic testing for hereditary deafness. Am J Hum Genet. 1998, 63: 1175-1180. 10.1086/302060.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Tan EJ, Tanner EK, Seeman TE, Xue QL, Rebok GW, Frick KD, Carlson MC, Wang T, Piferi RL, McGill S, et al: Marketing public health through older adult volunteering: experience corps as a social marketing intervention. Am J Public Health. 2010, 100: 727-734. 10.2105/AJPH.2009.169151.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Andreasen AR: Marketing Social Change: Changing Behaviors to Promote Health, Social Development, and the Environment. 1995, San Francisco: Jossey-Bass

    Google Scholar 

  25. Kotler P, Roberto N, Lee N: Social Marketing: Improving the Quality of Life. 2002, Thousand Oaks, CA: Sage Publications, 2

    Google Scholar 

  26. Tamaskar P, Malia T, Stern C, Gorenflo D, Meador HE, Azaove P: Preventive attitudes and beliefs of deaf and hard-of-hearing individuals. Arch Fam Med. 2000, 9: 518-525. 10.1001/archfami.9.6.518.

    Article  CAS  PubMed  Google Scholar 

  27. Polonsky MJ, Brijnath B, Renzaho AMN: “They don’t want our blood”: social inclusion and blood donation among African migrants in Australia. Soc Sci Med. 2011, 73: 336-342. 10.1016/j.socscimed.2011.05.030.

    Article  PubMed  Google Scholar 

  28. Toomey P, Hanlon N, Bates J, Poole G, Lovato CY: Exploring the role of social capital in supporting a regional medical education campus. Rural Remote Health. 2011, 11: 1774-

    PubMed  Google Scholar 

  29. Lomas J: Social capital and health: implications for public health and epidemiology. Soc Sci Med. 1998, 47: 1181-1188. 10.1016/S0277-9536(98)00190-7.

    Article  CAS  PubMed  Google Scholar 

  30. Boudreault P, Baldwin EE, Fox M, Dutton L, Tullis L, Linden J, Kobayashi Y, Zhou J, Sinsheimer JS, Sininger Y, et al: Reasons for genetic teting for deafness within the deaf community depend on cultural affiliation: results from a prospective, longitudinal genetic counseling and testing study. J Deaf Stud Deaf Edu. 2010, 15: 209-227. 10.1093/deafed/enq012.

    Article  Google Scholar 

  31. Baldwin EE, Boudreault P, Fox M, Sinsheimer JS, Palmer CGS: Effect of pre-test genetic counseling for deaf adults on knowledge of genetic testing. J Genet Counsel. 2012, 21: 256-272. 10.1007/s10897-011-9398-1.

    Article  Google Scholar 

  32. Palmer CGS, Boudreault P, Baldwin EE, Fox M, Deignan JL, Kobayashi Y, Sininger Y, Grody WW, Sinsheimer JS: Deaf genetic testing and psychological well-being. J Genet Counsel. 2013, 22: 492-507. 10.1007/s10897-013-9573-7.

    Article  Google Scholar 

  33. Glanz K, Lewis FM, Rimer BK: Health Behavior and Health Education: Theory, Research and Practice. 2002, San Francisco: Jossey-Bass, 3

    Google Scholar 

  34. Kotler P, Roberto EL: Social Marketing Strategies for Changing Public Behavior. 1989, Canada: Free Press

    Google Scholar 

  35. Lefebvre RC, Flora JA: Social marketing and public health intervention. Health Educ Quart. 1988, 15: 299-315. 10.1177/109019818801500305.

    Article  CAS  Google Scholar 

  36. McKenzie JF, Neiger BL, Smeltzer JL: Planning, Implementing, and Evaluating Health Promotion Programs: A Primer. 2005, San Francisco: Pearson Benjamin Cummings, 4

    Google Scholar 

  37. Walsh DC, Rudd RE, Moeykens BA, Moloney TW: Social marketing for public health. Health Affair. 1993, 12: 104-119. 10.1377/hlthaff.12.2.104.

    Article  CAS  Google Scholar 

  38. Weinreich NK: Hands on Social Marketing: A Step by Step Guide. 1999, Thousand Oaks, CA: Sage

    Google Scholar 

  39. Kotler P, Lee NR: Social Marketing: Influencing Behaviors for Good. 2008, Thousand Oaks, CA: Sage Publications

    Google Scholar 

  40. Buchbinder M, Timmermans S: Newborn screening and maternal diagnosis: rethinking family benefit. Soc Sci Med. 2011, 73: 1014-1018. 10.1016/j.socscimed.2011.06.062.

    Article  PubMed  Google Scholar 

  41. Lieu CC, Sadler GR, Fullerton JT, Stohlmann PD: Communication strategies for nurses interacting with patients who are deaf. Dermatology Nurs. 2007, 19: 541-544. 549–551

    Google Scholar 

  42. NAD-RID Code of Professional Conduct.,

  43. Daniel KL, Bernhardt JM, Eroglu D: Social marketing and health communication: from people to places. Am J Public Health. 2009, 99: 2120-2122. 10.2105/AJPH.2009.182113.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Mattson M, Basu A: The message development tool: a case for effective operationalization of messaging in social marketing practice. Health Market Quart. 2010, 27: 275-290. 10.1080/07359683.2010.495305.

    Article  Google Scholar 

  45. Arnos K, Israel J, Cunningham M: Genetic counseling of the deaf. Medical and cultural considerations. Ann NY Acad Sci. 1991, 630: 212-222. 10.1111/j.1749-6632.1991.tb19590.x.

    Article  CAS  PubMed  Google Scholar 

  46. Baker C: Foundations of Bilingual Education and Bilingualism. 2001, Cleveland: Multilingual Matters, 3

    Google Scholar 

  47. Ancker JS, Carpenter KM, Greene P, Hoffman R, Kukafka R, Marlow LAV, Prigerson HG, Quillin JM: Peer-to-peer communication, cancer prevention, and the internet. J Health Commun. 2009, 14: 38-46. 10.1080/10810730902806760.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Cialdini RB: Influence: Science and Practice. 2009, Boston, MA: Pearson Education, 5

    Google Scholar 

  49. Holdford DA: Using buzz marketing to promote ideas, services, and products. J Am Pharm Assn. 2004, 44: 387-395.

    Article  Google Scholar 

  50. SAS: SAS. 2002, Cary, North Carolina: SAS Institute Inc, 91

    Google Scholar 

  51. Enns EE, Boudreault P, Palmer CGS: Examining the relationship between genetic counselors’ attitudes toward deaf people and the genetic counseling session. J Genet Counsel. 2010, 19: 161-173. 10.1007/s10897-009-9272-6.

    Article  Google Scholar 

  52. Zazove P, Niemann LC, Gorenflo DW, Carmack C, Mehr D, Coyne JC, Antonucci T: The health status and health care utilization of deaf and hard-of-hearing persons. Arch Fam Med. 1993, 1993 (2): 7-

    Google Scholar 

  53. Zazove P, Meador HE, Reed BD, Sen A, Gorenflo D: Cancer prevention knowledge of people with profound hearing loss. J Gen Intern Med. 2009, 24: 320-326. 10.1007/s11606-008-0895-3.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Kusters A: Deaf utopias? Reviewing the sociocultural world’s “Martha’s Vineyard Situations”. J Deaf Stud Deaf Edu. 2010, 15: 3-16. 10.1093/deafed/enp026.

    Article  Google Scholar 

  55. Barnett S, Franks P: Smoking and deaf adults: associations with age at onset of deafness. Am Ann Deaf. 1999, 144: 44-50. 10.1353/aad.2012.0120.

    Article  CAS  PubMed  Google Scholar 

  56. Woodward J: Implications for sociolinguistic research among the deaf. Sign Lang Stud. 1972, 1 (1): 1-7. 10.1353/sls.1972.0004.

    Article  Google Scholar 

Pre-publication history

Download references


Thanks to the Deaf, deaf and hard-of-hearing individuals of California for their participation in this research. This research was supported by the National Human Genome Research Institute (Ethical, Legal, and Social Issues Branch) [R01 HG003871] and manuscript preparation was supported in part by National Cancer Institute [R25 CA154290].

Author information

Authors and Affiliations


Corresponding author

Correspondence to Christina GS Palmer.

Additional information

Competing interests

The authors have no competing interests to declare.

Authors’ contributions

YK conceptualized this manuscript, contributed to the analysis and interpretation of data, and wrote the first draft. PB contributed to the analysis and interpretation of data and to revisions of the manuscript. KH carried out the statistical analysis. JS contributed to the analysis and interpretation of data and to revisions of the manuscript. CP made contributions to the conception and design of this study, to the analysis and interpretation of data and to revisions of the manuscript. All authors read and approved of the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2

Authors’ original file for figure 3

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Kobayashi, Y., Boudreault, P., Hill, K. et al. Using a social marketing framework to evaluate recruitment of a prospective study of genetic counseling and testing for the deaf community. BMC Med Res Methodol 13, 145 (2013).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: