- Open Access
A photograph of the researcher on the invitation letter did not affect the participation rate of a postal survey: a randomized study within a trial (SWAT)
BMC Medical Research Methodology volume 22, Article number: 249 (2022)
Participant recruitment is one of the main challenges in research. It is suggested that including researcher photographs might increase participation rates, but empirical evidence is lacking. This study within a trial (SWAT) aims to assess whether invitation letters including researcher photographs increase the participation rate in the context of a survey on medical second opinions.
Through 25 local register offices in Berlin and Brandenburg (Germany), we identified a random sample of 9990 persons. We randomly assigned our sample to the intervention group (IG) receiving an invitation letter with researcher photographs and control group (CG) receiving an invitation letter without photographs in a 1:1 ratio. Our primary outcome was the participation rate. Furthermore, we compared participants to non-participants’ characteristics.
Of 9990 invitations, 9797 could be delivered (IG: 4890, CG: 4907). Of these, 1349 (13.8%) participated. There were 682/4890 (13.9%) participants in the IG and 662/4907 (13.5%) in the CG with an odds ratio of 1.030 (95% confidence interval: 0.918–1.156). Additional analyses on non-participant characteristics did not show any differences.
We could not find any difference in the participation rates. Our study does not confirm the results of previous studies. The length of our questionnaire may have affected our results.
Queens University Belfast – SWAT Store, SWAT 104.
Recruitment is one of the main issues in any research that depends on participant involvement. Only 55% of trials recruited their estimated sample size according to two funding agencies from the United Kingdom . Unsurprisingly, “Research into methods to boost recruitment in trials” was rated to be the top priority research in a survey on methodological research prioritization among directors of clinical trial units in 2012 . The problem of recruitment does also arise in survey research. There are numerous ways of enhancing the participation rate in surveys . Effectiveness of these enhancement strategies does not only rely on the type of survey (postal, by phone or electronic) but also on the sender or the receiver of the survey. In addition to monetary and non-monetary incentives in the form of direct compensation or the chance to win a lottery prize, various strategies might have a more subtle effect. The focus can be on the relationship with the sender, the length, appearance and style of the questionnaire, but also on low-threshold responses (e.g., stamped envelopes) [4,5,6,7,8,9]. Enhancement strategies that were found to be effective often result in high costs. It is most effective to pay prepaid cash incentives to the recipients of the survey . The use of stamped envelopes for reply instead of business-reply envelopes is more effective but leads to higher costs, too [11, 12]. In a university setting with low financial resources, appearance of the questionnaire, envelope or invitation letter and building up a relationship between sender and receiver through personalization might be a solution. Personalization can be a personalized salutation or a handwritten signature, but also a photograph or personal note of the researcher . Nevertheless the design should prevent that those who choose to participate in a survey may have systematic different characteristics (including attitudes toward participating in a survey) than those who refuse to participate to avoid selection bias .
A thorough Cochrane review found that there was convincing evidence only for the length of a questionnaire (short questionnaires increase participation rates), but not for the effects of other strategies . Hence, we chose a personalization strategy, which does not cost additional money and other resources, to analyze the effect on the participation rate in a large sample. Our sample was drawn from the general population. The aim of this study within a trial (SWAT) was to compare the participation rate resulting from different ways to contact the participants (photograph vs. no photograph) in a survey regarding attitudes towards medical second opinions and whether there are sociodemographic subgroups for which the intervention has a larger effect.
This SWAT was registered in Queens University Belfast – SWAT Store, as SWAT 104 (04/10/2019).
In the context of a population based survey on attitudes of the general German population towards medical second opinions we conducted a SWAT using a parallel group, randomized controlled trial with a 1:1 allocation. For details please consult the original paper . The survey is part of a mixed method study on medical second opinions in Germany . We describe the design of the original study in brief: We randomly identified 9990 individuals via 25 randomly selected population registration offices in the states of Berlin and Brandenburg. We selected these two states because they cover a settlement structure with both very high but also very low population density. Participants were selected through a disproportionate stratified sample, with settlement structure (cities, towns and suburbs, rural areas) serving as the stratification variable, resulting in three equally sized samples. Settlement patterns are defined by “degree of urbanization” of Eurostat . The register offices belonged to cities (n = 5), towns and suburbs (n = 10) and rural areas (n = 10). There are only 5 urban areas in those federal states, which is why we chose all of them. See Fig. 1 for the detailed process. Moreover the register offices contained data on gender and most on age as well. The questionnaire contained 47 items on 14 pages. It included 8 parts: health-related items (2 items), local medical care situation (5 items), patients’ needs concerning second opinions (7 items), potential experiences with second opinions (8 items), design of the second opinion procedure (8 items), experiences with and knowledge of second opinion programs by health insurers (3 items), experiences with and knowledge of the offer of second opinion provider (3 items). The last part included sociodemographic characteristics (11 items). We used the European Health Literacy Survey including 16 items to assess health literacy .
As this was a population based survey we had no eligibility criteria except age (≥18 years) and residence in either Berlin or Brandenburg. The questionnaire was send via post twice (initially and 6 weeks later with a reminder) together with the invitation to participate. The questionnaire could be send back to us with business-reply envelopes. By returning the questionnaire to us, participants could also include their declaration of consent to take part in a lottery to win one of 125 Amazon vouchers (50€ each) in a separate envelope.
The intervention group received an invitation letter with a colored photograph of two female researchers, and the control group received an invitation letter without a photograph. Otherwise the letters were identical. Please see Additional file 1 for details.
Originally, we planned to perform a factorial design with two different interventions in a 1:1:1:1 manner. The second intervention was a teaser on the envelope (“Join now and help shape healthcare”) compared to an unprinted envelope. Unfortunately, the printing company only sent unprinted envelopes, so that we did not follow up on this.
The outcome was the participation rate. The participation rate is defined as the number of people who answered at least one question relative to the total number of those who received an invitation to the survey. Whereas response rate is defined as the number of people who respond to the survey in any way, relative to the total number of recipients of the survey invitation. The response rate can always be higher than the participation rate a refusal to participate would be a response but not a participant.
All of the addressees of our survey on second medical opinions were included in this SWAT. We chose an equally sized disproportionate stratified sample (cities, towns and suburbs, rural area) as we hypothesized that there might be differences regarding our initial study question (second opinions) between the settlement patterns.
For randomization, one researcher used random numbers generated by Microsoft Excel. Participants were originally allocated in a 1:1:1:1 manner without stratification. In group 1: cover letter without photograph and without a teaser on the envelope, group 2: cover letter with photograph without a teaser on the envelope, group 3: cover letter without photograph with a teaser on the envelope, group 4: cover letter with photograph with a teaser on the envelope. The same researcher who determined the allocation sent the address data of the four groups to the printing company. As soon as we found out that the printing of the envelopes with a teaser had failed, we merged group 1 and group 3 to group 1 and group 2 and group 4 to group 2.
The participants were not aware that we conducted this SWAT within our survey. There was only a small mark on the bottom of the questionnaire “I”, “II”, “III”, “blank space “for the groups 1, 2, 3 and 4 so that researchers could identify in which group the participants were allocated. The researchers were not blinded for data analysis.
All data received in the survey were extracted in an excel spreadsheet. And data form the whole sample, received from the registration offices (gender, postal code, and age for most of the population registration offices), were added to a second excel spreadsheet. We categorized postal codes according to the ‘degree of urbanisation’ of Eurostat and regional statistics by the German Federal Statistical Office, which are in accordance with the settlement patterns described earlier [17, 19]. To create international comparability we used the Comparative Analysis of Social Mobility in Industrial Nations (CASMIN) categorization to abstract education. CASMIN contains three main categories, primary, secondary and tertiary education with various subcategories. We used the categories tertiary education (academic degree, independent of school education and type of degree), higher secondary (no or any vocational education with at least intermediate school education) and lower secondary (no or any vocational education with at least general school education) . Due to the study design, we were not able to collect the reasons for non-participation systematically.
We calculated the risk difference and the odds ratio for the primary outcome using contingency tables. Differences between characteristics of participants in both groups were tested by performing χ2 analyses for binary data and t-test for continuous data. We performed exploratory subgroup analyses in women, men, those who lived in cities, in towns and suburbs and in rural areas, comparing participants and non-participants.
In the subgroup analyses we also used χ2 analyses for binary data. We planned to use logistic regressions but were not able to do so, as individual data of participants was not fully assignable. All analyses were performed in SPSS version 28.0 and Microsoft Excel.
On the 6th of March 2020 we sent 9990 invitations and the reminders on the 24th of April. Of the initial 9990 participants 193 could still not be reached after 2 attempts, reasons were for example “receiver unknown” or “moved without possibility to forward to new address”. We assumed that the questionnaire could be delivered to N 9797 persons, since it was not returned. However, we could not be sure if it was successfully received. See Table 1 for baseline characteristics.
Of these 9797, 1349 persons participated in our survey (participation rate of 13.8%, 1349/9797), see flow chart in Fig. 1. We received the last envelope in July 2020. In the group “with photograph” 682 (13.9%, 682/4890) people participated and in the group “without photograph” 667 (13.6%, 667/4907) people participated resulting in an absolute risk difference of − 0.36% (95% CI -1.719-1.011) and an odds ratio (OR) of 1.030 (95% CI 0.918–1.156). There was no relevant difference between the intervention group and the control group in any of the queried characteristics. See Table 2 for details on characteristics of participants.
Within a subgroup analyses of women, men, those who lived in cities, in towns and suburbs and in rural areas, we compared participants and non-participants. We saw neither in women nor in men a difference in participation. Furthermore, the participation rate was similar regarding settlement pattern. Even though we measured a little higher participation rate in the group with photograph in people living in towns and cities and a little lower participation rate in people living in rural areas, there was no relevant difference, see Table 3 for further details.
We conducted a SWAT to compare the effectiveness of invitation letters with a photograph of the researcher with that of invitation letters without a photograph of the researcher, as part of a survey of attitudes towards medical second opinions. We found no difference in the participation rate due to the use of a photograph. We neither found any differences when we compared gender and settlement patterns of participants and non-participants. We also planned to examine another intervention, a teaser “Join now and help shape healthcare” printed on the envelope. Unfortunately, all envelopes were sent out without this teaser by the printing company. Overall, the implementation of the SWAT caused no additional costs and very little additional work for the preparation of the invitation letter and the teaser.
In two other studies which used a photograph of the researcher on the invitation letter, results were contradictory. While one study reported a participation rate of29.3% and an OR of 2.9 (95% CI 1.5–6.1) for the photograph group (40.0% participation versus 19.0% participation), the other study reported in total 54.0% participation rate with 30.0% in both groups before a reminder and 50.0% in the group with photograph and 63.0% without photograph after a reminder [21, 22]. Both studies differed to ours in various aspects. Dommeyer et al. from 1997 surveyed 150 persons on music censorship using a 4-page questionnaire. The invitation letter was only half a page and was written from the perspective of a college student. 44 other college students rated the photo for physical attractiveness and sex appeal before the survey . In 1984, Rucker et al. asked 349 graduate students from the same university to answer a questionnaire on furniture. They used either a picture of the researcher in a formal dress, in casual clothes or no picture at all . Not only the setting of both studies and content of the surveys differed but also the methods used. Therefore, the studies are difficult to compare amongst each other as well as with our work. However, there are various reasons possible why results differed that much; one is the rating of attractiveness of the picture utilized by Dommeyer et al.. Gueguen et al. showed that the use of an attractive photograph increases the respone rate compared to a medium attractive photograph in an email survey . In Rucker et al. the personalization is generally higher, since graduates from the same university were contacted and all addresses were handwritten on the envelopes, which is expected to increase the participation rate . The authors assumed that there is a point of “overpersonalization”. They state that subjects reacted negatively (with non-participation) to the higher personalized conditions, especially after the reminder was sent, and assume it was “too much” . Another possible explanation is that the attitude towards personalization through a photograph has transformed over the last 30 years, as communication changed a lot and the use of photographs is nowadays familiar. Moreover, our sample was much larger than that of both other studies. This might be due to the fact that their samples were too small to show the true effect. According to the Leverage Salient Theory, individuals assign varying importance to different aspects of a survey. Some are attracted by the topic, some by the sender and some by the incentive . If researchers put an emphasis on one of these aspects, they may influence who will participate to a small extent. The emphasis cannot be generalized because the value of each aspect varies in a population . Given our results and the findings from the Cochrane review by Edwards et al., the question of individualized recruitment strategies rather than a one fits all solution is pressing. Future research should consider under what conditions and for whom a particular aspect of the survey has an impact with the aim to use this information in recruiting strategies . We did not aim to emphasize any particular aspect in our survey. As stated above, a non-monetary strategy is inexpensive and easy to implement. When conducting a study or survey one should always consider if a SWAT is possible and useful.
Our SWAT has several limitations. We assume that the low participation rate of 13.8% could be a factor that leads to the possible effects of a recruitment measure being overshadowed. One possible reason for the low participation rate is the length of our survey. Filling out our questionnaire took about 30 to 40 minutes. In a survey of physicians, Jepson et al. showed a decrease from a 60% response rate for questionnaires with 849 words to 16.7% for questionnaires with more than 1800 words . The participation rate tends to be higher when one is affected by a topic him- or herself or in general with a “more interesting” topic . Our target group was the general population and it was not clear how many people already had experience or interest in medical second opinions. However, it is a topic which may affect anyone. Furthermore, we used a mixture of attitudinal and factual questions. Cartwright et al. showed that factual questions lead to higher participation rates than mixed questions . As an incentive, we used a lottery. Those who participated had the chance to win a 50 Euro Amazon voucher (we raffled 125 vouchers). Interestingly, only 70% of the participants took part in the lottery. We expected a higher rate, but some people criticized the use of Amazon vouchers as they had objections against Amazon as a company. In principle, the use of prepaid cash incentives has shown to be the most useful means to increase participation rates . As our funding was limited, wewere not able to give a prepaid incentive to everyone. In addition, Gajic et al. found that although participation rates were higher for a prepaid incentive, the higher the lottery prize, the more participants returned completed questionnaires . Unfortunately, we were not able to check our second intervention, the teaser, because the printing company sent all the envelopes unprinted by mistake. Our expectation were higher participation rates through the use of the teaser . The first round of invitation letters was sent during the first weeks of a COVID-19 induced lockdown. It is unclear whether and how this affected our results. Furthermore, the timeframe between the first invitation and the reminder was 7 weeks. One reason for that long period between first contact and reminder was that we had about 400 invitations sent back to us because of an error in the addressees, which we sent out again. We wanted to be sure that there was an interval of at least 4 weeks between the first invitation and the reminder for all recipients. We have not found any evidence on what is the best period to send one or more reminders, but at least a recommendation to send the first reminder after three weeks and to send more than one reminder [31, 32].
In general, we were surprised that we did not find any other SWAT or other recent studies on the same topic, as recruitment is a highly relevant topic and performing a SWAT like we did is quite cost effective. We imply, considering our results, that there might be other SWATs on the topic resulting in no effect which were not published.
We did not find any difference in the participation rate when we used a photograph of the researcher in the invitation letter or chose not to do so. We neither saw any difference of sociodemographic factors in the non-participant analysis. More research on recruitment for surveys with low funding is needed. It is easy and inexpensive to add a photo of the researchers to the invitation letter when conducting a survey. Although we did not find an increase in participation rate, we also did not find any negative effects of using a photograph.
Availability of data and materials
The datasets generated and analysed during the current study will be available from the corresponding author on reasonable request.
Comparative Analysis of Social Mobility in Industrial Nations
Study within a trial
Sully BGO, Julious SA, Nicholl J. A reinvestigation of recruitment to randomised, controlled, multicenter trials: a review of trials funded by two UK funding agencies. Trials. 2013;14:166.
Tudur Smith C, Hickey H, Clarke M, Blazeby J, Williamson P. The trials methodological research agenda: results from a priority setting exercise. Trials. 2014;15:32.
Phillips AW, Reddy S, Durning SJ. Improving response rates and evaluating nonresponse bias in surveys: AMEE guide no. 102. Med Teach. 2016;38:217–28.
Mercer A, Caporaso A, Cantor D, Townsend R. How much gets you how much? Monetary incentives and response rates in household surveys. Public Opin Q. 2015;79:105–29.
Laguilles JS, Williams EA, Saunders DB. Can lottery incentives boost web survey response rates? Findings from four experiments. Res High Educ. 2011;52:537–53.
Sahlqvist S, Song Y, Bull F, Adams E, Preston J, Ogilvie D, et al. Effect of questionnaire length, personalisation and reminder type on response rate to a complex postal survey: randomised controlled trial. BMC Med Res Methodol. 2011;11:62.
Jobber D, Sanderson S. The effects of a prior letter and coloured questionnaire paper on mail survey response rates. J Mark Res Soc. 1983;25:339–49.
Mullner RM, Levy PS, Byre CS, Matthews D. Effects of characteristics of the survey instrument on response rates to a mail survey of community hospitals. Public Health Rep. 1982;97:465–9.
Groves BW, Olsson RH. Response rates to surveys with self-addressed, stamped envelopes versus a self-addressed label. Psychol Rep. 2000;86:1226–8.
Pforr K, Rammstedt B. Verwendung von geldwerten Belohnungen (Incentives) in persönlichen Umfragen vor dem Hintergrund sinkender Ausschöpfungsraten. Psychother Psychosom Med Psychol. 2016;66:280–2.
Wakabayashi C, Hayashi K, Nagai K, Sakamoto N, Iwasaki Y. Effect of stamped reply envelopes and timing of newsletter delivery on response rates of mail survey: a randomised controlled trial in a prospective cohort study. BMJ Open. 2012;2(5):e001181.
Edwards PJ, Roberts I, Clarke MJ, Diguiseppi C, Wentz R, Kwan I, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009;2009(3):MR000008.
Scott P, Edwards P. Personally addressed hand-signed letters increase questionnaire response: a meta-analysis of randomised controlled trials. BMC Health Serv Res. 2006;6:111.
Armstrong D, Ashworth M. When questionnaire response rates do matter: a survey of general practitioners and their views of NHS changes. Br J Gen Pract. 2000;50:479–80.
Könsgen N, Prediger B, Schlimbach A, Bora A-M, Hess S, Caspers M, et al. Attitude toward second opinions in Germany – a survey of the general population. BMC Health Serv Res. 2022;22:76.
Bruch D, May S, Prediger B, Könsgen N, Alexandrov A, Mählmann S, et al. Second opinion programmes in Germany: a mixed-methods study protocol. BMJ Open. 2021;11:e045264.
Eurostat. Territorial typologies manual 2018. [cited 2021 January 14th]; Available from: https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Territorial_typologies_manual.
Röthlin F, Pelikan J, Ganahl K. Die Gesundheitskompetenz der 15-jährigen Jugendlichen in Österreich. Abschlussbericht der österreichischen Gesundheitskompetenz Jugendstudie im Auftrag des Hauptverbands der österreichischen Sozialversicherungsträger (HVSV). 2013.
Statistisches Bundesamt (Destatis). Alle politisch selbständigen Gemeinden mit ausgewählten Merkmalen am 31.03.2020 (1). (Quartal) Statistisches Bundesamt (Destatis); 2020. [cited 2021 January 14th]; Available from: https://www.destatis.de/DE/Themen/Laender-Regionen/Regionales/Gemeindeverzeichnis/Administrativ/Archiv/GVAuszugQ/AuszugGV1QAktuell.html.
Brauns H, Scherer S, Steinmann S. The CASMIN educational classification in international comparative research. In: Hoffmeyer-Zlotnik JHP, Wolf C, editors. Advances in cross-National Comparison: a European working book for demographic and socio-economic variables. Boston, MA: Springer US; 2003. p. 221–44.
Dommeyer CJ, Ruggiero LA. The effects of a photograph on mail survey response. MARKETING BULLETIN-DEPARTMENT OF MARKETING MASSEY UNIVERSITY. 1996;7:51–7.
Rucker M, Hughes R, Thompson R, Harrison A, Vanderlip N. Personalization of mail surveys: too much of a good thing? Educ Psychol Meas. 1984;44:893–905.
Guéguen N, Legohérel P, Jacob C. Sollicitation de participation à une enquête par courriel: Effet de la présence sociale de l’attrait physique du demandeur sur le taux de réponse. [Solicitation of participation in an investigation by e-mail: Effect of the social presence of the physical attraction of the petitioner on the response rate]. Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement. 2003;35:84–96.
Groves RM, Singer E, Corning A. Leverage-saliency theory of survey participation: description and an illustration. Public Opin Q. 2000;64:299–308.
Zhang C, Lonn S, Teasley SD. Understanding the Impact of Lottery Incentives on Web Survey Participation and Response Quality:A Leverage-salience Theory Perspective. Field Methods. 2017;29:42–60.
Jepson C, Asch DA, Hershey JC, Ubel PA. In a mailed physician survey, questionnaire length had a threshold effect on response rate. J Clin Epidemiol. 2005;58:103–5.
Cartwright A. Some experiments with factors that might affect the response of mothers to a postal questionnaire. Stat Med. 1986;5:607–17.
Warriner K, Goyder J, Gjertsen H, Hohner P, Mcspurren K. Charities, no; Lotteries, no; cash, Yes: main effects and interactions in a Canadian incentives experiment. Public Opin Q. 1996;60:542–62.
Gajic A, Cameron D, Hurley J. The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey. Eur J Health Econ. 2012;13:789–99.
Dommeyer CJ, Elganayan D, Umans C. Miscellany: increasing mail survey response with an envelope tester. Market Research Society Journal. 1991;33:1–5.
Menold N. Postal Surveys (Version 2.0). (GESIS Survey Guidelines). Mannheim: GESIS - Leibniz-Institut für Sozialwissenschaften; 2016. https://doi.org/10.15465/gesis-sg_en_018.
Christensen AI, Ekholm O, Kristensen PL, Larsen FB, Vinding AL, Glümer C, et al. The effect of multiple reminders on response patterns in a Danish health survey. Eur J Public Health. 2014;25:156–61.
Other information and registration
Queens University Belfast – SWAT Store, SWAT 104.
The Protocol can be found under.
Open Access funding enabled and organized by Projekt DEAL. This SWAT did not receive any funding. But it was embedded in the ZWEIT project, funded through Innovation Committee of the Federal Joint Committee (Innovation Fund, grant number 01VSF18014).
Ethics approval and consent to participate
Our survey, including this SWAT, was approved by the ethics committee of Witten/Herdecke University (case number: 162/2019). The participants agreed to participate in the study by voluntarily completing the questionnaire and thereby gave their informed consent. All methods were carried out in accordance with relevant guidelines and regulations.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Prediger, B., Könsgen, N., Bora, AM. et al. A photograph of the researcher on the invitation letter did not affect the participation rate of a postal survey: a randomized study within a trial (SWAT). BMC Med Res Methodol 22, 249 (2022). https://doi.org/10.1186/s12874-022-01717-3
- Participation rate
- Enhancement strategies
- Study within a trial