Do personalised e-mail invitations increase the response rates of breast cancer survivors invited to participate in a web-based behaviour change intervention? A quasi-randomised 2-arm controlled trial
© Short et al. 2015
Received: 5 February 2015
Accepted: 4 August 2015
Published: 19 August 2015
The Erratum to this article has been published in BMC Medical Research Methodology 2015 15:102
Previous research has shown that the personalisation of study invitations improves response rates in survey-based research. To examine if this finding extends to experimental studies, we examined the impact of personalised study invitation e-mails on the response rates of potentially eligible breast cancer survivors for participation in a 6 month randomised controlled trial testing the efficacy of a physical activity intervention.
Potential participants (n = 344) were sent either a personalised email or a generic email.
Those sent the personalised email were 1.5 times (95 % CI = 1.18–1.93) more likely to respond than those sent the generic email.
These findings suggest that personalisation may be a useful and potentially powerful tool that can be utilised when recruiting participants into experimental studies in order to boost response rates.
The potential of online behaviour change interventions for improving public health in both a primary and tertiary prevention setting is well recognised [1, 2]. Due to this, and the growing popularity of the internet, the last decade has seen a substantial increase in the number of online behaviour change interventions developed and evaluated [3, 4]. Several well-conducted systematic reviews and meta-analyses have synthesised the literature regarding this research [1, 3, 5–9]. In general, these reviews have shown that online interventions can be effective (albeit effect sizes have been small), but that issues with recruiting and retaining participants are commonly reported [1, 3, 5–9]. If online interventions are to be an effective public health tool, efforts to address these issues are needed so that interventions have sufficient reach to have a real-world impact .
The personalisation of study materials (where individuals are referred to by their name) is one technique that has been found to increase response rates [11–14]. Compared to generic invitations to engage in research, the receipt of personalised materials has been shown to increase participation, including in studies conducted in an online environment [11–14]. However, this research has been restricted to survey research, predominately of a cross-sectional nature. For studies that require a far greater deal of commitment on behalf of the participant, such as randomised controlled trials, personalisation of study materials may not result in increased participation rates. In some circumstances, personalisation of recruitment materials for randomised controlled trial may not be possible, due to a lack of personal information available at the time of recruitment. However, in other circumstances, for example in studies targeting individuals with chronic diseases (where national registries containing contact information exist) or in studies employing setting-based recruitment methods (such as workplaces) it may be possible to personalise recruitment materials and doing so may help to boost response rates [11–14].
This study aimed to examine the impact of personalised study invitation emails on the response rates of potentially eligible breast cancer survivors for participation in a randomised controlled trial. The randomised trial itself examined the relative efficacy of three online computer-tailored interventions (differing in delivery schedule only) designed to increase participation in physical activity over a three month period. The trial period was for 6 months, with assessments occurring at baseline, 3 months and 6 months post-baseline. Participation was entirely by distance with all assessments conducted online via the study website.
The current study is a nationally-based quasi-randomised 2-arm controlled trail. Ethical approval was obtained from the Human Research Ethics Committee at Central Queensland University, Australia (H13/07-126). The trial is registered with the Australian and New Zealand Clinical Trials Registry (registration number: ACTRN12613001220752).
Participants and procedure
The study was conducted between June and August 2014 at Central Queensland University. English proficient female breast cancer survivors who were over 18 years of age, who had finished active cancer treatment, had no contraindications to exercise and were not already participating in 150 min of moderate-vigorous aerobic activity accumulated across at least 5 days a week were eligible to participate. To recruit participants, members (n = 43,150) of a breast cancer organisation in Australia, who had agreed to be contacted about breast cancer- related research opportunities, were e-mailed by the organisation about the upcoming randomised trial on behalf of the research team. Volunteers include both male and female member of the general community. The e-mail contained information about the study (study aims, procedures and eligibility criteria) and an instruction to complete a ‘permission to pass on contact details form’ if they thought they were eligible and agreed to be contacted by the research team. No reminder emails were sent.
The breast cancer organisation provided the research team with the contact information of members who had agreed to be contacted in relation to the randomised controlled trial. This information was provided once a week over an eight week period, as not all members responded equally fast to the request. Upon receiving the contact information each week the research team sent one group a personalised email (i.e., addressing them with first and last name) and the other half a generic email (i.e., addressing them as ‘dear member’). The email detailed the study aims, the study eligibility criteria and directed participants to the study website where they could view more information about the study and consent to participate by completing the eligibility questionnaire. The greeting line (personalised or generic) was the only difference between the e-mails sent to each group (see Additional file 1). All participants were sent a reminder email 2–3 weeks after the initial e-mail was sent, which was personalised in the same way as the original e-mail.
Group allocation was based on last name. Each week the contact information of potential participants was provided to the research team in an excel spreadsheet. This was sorted based on last name and split into two groups (personalised email versus generic email). To reduce any potential bias associated with this, the group receiving the personalised e-mail was switched each week (i.e., in the first week the personalised email was sent to the first half of participants in the excel sheet and in the second week the personalised email was sent to the second half of participants in the excel sheet, and so on for proceeding weeks). All participants were blinded to this process. The project team was not blinded.
Potential participants were classified as ‘responders’ if they completed the eligibility questionnaire. This was determined by cross-referencing contact details (i.e., full name and email address) collected during eligibility screening with information provided by the breast cancer organisation. To explore the association between personalisation and the likelihood of responding a chi-square analysis was performed in Stata using the cs epitab command, which computes both the test statistic and risk ratio.
Previous research has shown that the personalisation of study invitations significantly increases participant response rates to epidemiology-based research [11–14]. In the current study, a similar effect was observed, even though the level of commitment requested from participants was markedly higher compared to the previous studies. An important difference in response rate was observed (i.e., 1.5 times higher), even though all participants had already received one e-mail about the study from the breast cancer organisation and the sample contained those already interested in breast cancer related research. Taken together, these findings suggest that personalisation may be a useful and potentially powerful tool that can be utilised when recruiting participants into epidemiological and experimental studies in order to boost response rates.
There are some limitations of the study that should be considered when interpreting the results. First, for convenience, the method of allocation used was not truly random and may have introduced unintentional bias. For example, assigning participants based on last name can result in some ethic groups being disproportionately assigned . We did attempt to reduce the potential of this occurring by switching the block of participants that received the personalised email each week, however some bias may still have been introduced. Second, the research team was not blinded to group allocation. While this introduces another potential source of bias , it is proposed that this is unlikely an issue in the current study given that all potential participants were unknown to the research team, there was no face-to-face contact and the outcome measure was objective (i.e., whether the eligibility questionnaire was submitted) and not open to interpretation. Of note, participants were blinded to group allocation and not aware of the study aims, which is a strength of the study. Finally, due to privacy restrictions we do not have participant characteristic data for non-responders or those deemed ineligible to participate, and as a result are unable to determine the number of potentially eligible non-responders or explore differences in response rates based on participant characteristics. This data would be useful for examining the generalizability of our findings, especially given that very few (1.8 %) of the potential participants contacted by the breast cancer organisation agreed to be contacted by the research team. While this low participation rate was likely partially due to ineligibility (since organisation list members included both men and women and people with and without a history of cancer) there may have been significant differences between responders and non-responders that were eligible and this would have implications for the generalizability of our findings. Nonetheless, the findings do provide evidence that personalisation was a useful tool for recruiting eligible participants in our study. Of note, no differences in eligibility were found based on whether a personalised invitation was sent and the vast majority of participants that completed the eligibility questionnaire (91 %) were deemed eligible and went on to participate in the RCT. Thus, given that personalisation significantly increased the odds of completing the eligibility questionnaire, it ultimately increased the odds of participating in the trial.
In light of our research findings it is recommended that researchers consider if the personalisation of study materials is both possible and appropriate for enhancing recruitment in their studies. Unfortunately, personalisation is not possible when names from the sampling frame are not available or are unobtainable. Further, personalisation may not be appropriate when the researcher has no known relationship to the potential participants or when the subject matter is of a sensitive nature . In the case of the former, introduction by a known third party is recommended.
As personalisation of recruitment materials may help to increase response rates this strategy should be considered when recruiting participants into randomised trials.
CES holds a Post-Doctoral Fellowship funded through National Health and Medical Research Council of Australia. CV holds a Future Leader Fellowship funded through the National Heart Foundation of Australia.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Murray E. Web-based interventions for behavior change and self-management: potential, pitfalls, and progress. J Med Internet Res. 2012;14(4):30–30.Google Scholar
- Kuijpers W, Groen WG, Aaeronson NK, van Harten WH. A systematic review of web-based interventions for patient empowerment and physical activity in chronic diseases: relevance for cancer survivors. J Med Internet Res. 2013;15(2), e37.View ArticlePubMedPubMed CentralGoogle Scholar
- Ritterband LM, Tate DF. The science of internet interventions. Introduction. Ann Behav Med. 2009;38(1):1–3.View ArticlePubMedGoogle Scholar
- Lustria ML, Noar SM, Cortese J, Van Stee SK, Glueckauf RL, Lee J. A meta-analysis of web-delivered tailored health behavior change interventions. J Health Commun. 2013;18(9):1039–69.View ArticlePubMedGoogle Scholar
- Webb T, Joseph J, Michie S. Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res. 2010;12(1), e4.View ArticlePubMedPubMed CentralGoogle Scholar
- Davies C, Spence JC, Vandelanotte C, Caperchione CM, Mummery KW. Meta-analysis of internet-delivered interventions to increase physical activity levels. Int J Behav Nutr Phys Act. 2012;9(1):52.View ArticlePubMedPubMed CentralGoogle Scholar
- Morrison LG, Yardley L, Powell J, Michie S. What design features are used in effective e-health interventions? A review using techniques from critical interpretive synthesis. Telemed J E Health. 2012;18(2):137–44.View ArticlePubMedGoogle Scholar
- Brouwer W, Kroeze W, Crutzen R, Nooijer J, de Vries NK, Brug J, et al. Which intervention characteristics are related to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res. 2011;13(1), e2.View ArticlePubMedPubMed CentralGoogle Scholar
- Joseph RP, Durany NH, Benitez TJ, Pekmezi DW. Internet-based physical activity interventions. Am J Lifestyle Med. 2014;8(1):42–67.View ArticlePubMedGoogle Scholar
- Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: The RE-AIM framework. Am J Public Health. 1999;89:1322–7.View ArticlePubMedPubMed CentralGoogle Scholar
- Dillman DA, Lesser V, Mason R, Carlson J, Willits F, Robertson R, et al. Personalization of mail surveys for general public and populations with a group identity: results from nine studies*. Rural Sociol. 2007;72(4):632–46.View ArticleGoogle Scholar
- Heerwegh D. Effects of personal salutations in E-mail invitations to participate in a web survey. Public Opin Q. 2005;69(4):588–98.View ArticleGoogle Scholar
- Muñoz-Leiva F, Sánchez-Fernández J, Montoro-Ríos F, Ibáñez-Zapata JA. Improving the response rate and quality in Web-based surveys through the personalization and frequency of reminder mailings. Quality Quantity. 2010;44(5):1037–52.View ArticleGoogle Scholar
- Sauermann H, Roach M. Increasing web survey response rates in innovation research: an experimental study of static and dynamic contact design features. Res Policy. 2013;42(1):273–86.View ArticleGoogle Scholar
- Viera AJ, Bangdiwala SI. Eliminating bias in randomized controlled trials: importance of allocation concealment and masking. Fam Med. 2007;39(2):132–7.PubMedGoogle Scholar