Modes of contact for community-based epidemiological research traditionally include mail-outs to residential addresses, and telephone contact based on public directories or random-digit-dialing (RDD) to landline telephone numbers. In addition to deciding on contact mode(s), a choice needs to be made between personalised (directed to a named person) or generic (to the Householder/Occupant) approaches. However, the population coverage of personalised data sources based on public telephone directory listings has declined in recent years due to the rapid transition to wireless (mobile/cell) phones [1, 2], and higher proportions of unlisted landline numbers [3]. The rising prevalence of households with only wireless telephone connections has also impacted adversely on the coverage of generic RDD methodology for landline telephone numbers [2].
In addition, researchers are encountering greater problems in making contact with potential respondents and rising refusal rates among those contacted [4]. Declining response rates have been reported for long established population health surveys including the U.S. Behavioral Risk Factor Surveillance Study (BRFSS), as well as public opinion and consumer surveys [4]. A review of the RDD monthly Survey of Consumer Attitudes by the University of Michigan [5] showed non-contacts accounted for less than 5% of non-response between 1979 and 1985, but increased to more than 15% by 2003. The rate of refusals among contacted households also increased from 19% in 1979 to 27% in 2003, and the overall response rate fell from 72% to 48%. The dual challenges of shrinking population coverage of traditional sampling frames and falling response rates underline the need to explore new sampling frames and alternative modes of data collection for community-based surveys and to document their utility.
Use of the internet for community surveys
With internet access now widespread in many countries, electronic methods of contact and data gathering for surveys have become possible. In common with telephone interviews, internet surveys offer many advantages for improving data quality compared to written questionnaires [6]. These include the ability to require or prompt for answers to missed questions, application of input masks and consistency checks to minimise invalid responses, and skipping of irrelevant sections which are conditional on the answers to previous questions. Problematic questions can be quickly identified by tracking completion times and break-off points, then modified to reduce the rate of incomplete surveys [7]. Initial costs for internet surveys vary depending on the level of programming sophistication required, but as data entry is performed by the respondents, the cost per response declines as the number of respondents increases. This offers potential savings compared to postal or telephone survey modes where material and staff costs tend to be proportional to respondent numbers.
For surveys of specialised populations such as students or professional groups it is usually possible to send an email invitation with a direct hyperlink to the internet survey to an existing email list, but the lack of such lists for the general population means that other methods of contact need to be used. Comparisons of internet surveys with other modes of survey administration have generally used conventional mail for initial contact, and a common finding has been that response rates to community internet surveys are lower than for postal surveys [8–10]. Two independent meta–analyses found that the average differential between internet surveys and other response modes was greater for community surveys than for surveys involving specific target groups such as students, employees or association members who may be more familiar with internet use [11, 12].
Even in countries such as Sweden where more than 80% of adults have internet access at home and there is high penetration across different age groups and education levels [9], it appears most people still prefer mail surveys. For example, a community internet health survey with one mail and one telephone follow up contact following a mailed invitation, obtained a response rate of 50.6%, but the parallel postal survey arm yielded a response rate of 64.4% [13]. Similarly, a lifestyle survey of Swedish women produced an initial response rate of 33% for a postal invitation to complete an internet survey [14], but when non-respondents were followed up by letter or email and offered the choice of postal or internet response, over 90% of those who responded at this stage chose the postal option.
Although internet surveys currently appear less effective for community studies than other response modes, the combination of multiple modes may offer a means to improve overall survey response rates and possibly broaden population coverage [15]. Indeed, it has been argued that use of any single mode of contact other than face-to-face household interviews will currently exclude significant sub-groups within the population [16]. Studies offering the internet mode simultaneously with other response modes have generally found little improvement in overall response rates [17, 18], but sequential use of different modes has been reported to be beneficial [15]. For example, a US community study using a generic address-based postal sampling frame found that response rates between 44% and 52% could be achieved using an initial internet survey with a pre-paid $5 cash incentive followed by a postal survey of non-responders [19]. Respondents to mail-only and internet+mail surveys were similar to each other in demographic characteristics and more closely matched population demographics than internet-only respondents. However, it was found that the mail-only comparison arms had higher response rates and lower costs, and the need to use postal contact for invitations and follow up negated the potential cost and speed advantages of the internet mode. In more restricted target group (subscribers to long distance telephone services), follow up of non-responders to four initial survey modes (mail, telephone, interactive voice response, or internet) using a different mode (telephone or mail) resulted in an increase of up to 37% in the overall response rate [20].
Investigation of factors affecting internet response rates among those who are successfully contacted shows general similarities to those documented for other survey modes [21]. The salience of the topic to the individual, the length and complexity of the survey and the type of sponsoring organisation (academic, government or commercial) all may influence a potential respondent’s decision to take part. Access to the internet is of course a prerequisite for responding, and the constraints of literacy and language apply for the internet mode as they do for mailed surveys, although the ability to provide access to questionnaires in alternative languages is greatly simplified on the internet.
The effect of personalisation and incentives
Personalisation has generally been found to increase response rates for postal questionnaires, but a recent systematic review showed a positive effect in only one of six randomised studies that used community-based sampling frames [22]. Another analysis of 17 comparisons involving the general public in the US found a modest benefit of personalisation on mail survey response rates with greater effects apparent in rural areas [23]. Personalisation of email invitations for internet surveys has been reported to increase response rates in a number of groups including university students [24, 25] and scientists and engineers [26]. This approach is not open to community surveys due to the lack of email lists, but personalisation of invitation letters for internet surveys might reasonably be expected to improve response rates compared to generic invitations.
Incentives for participation in surveys may be provided in monetary or non-monetary form, and paid unconditionally (sent to all potential respondents with the invitation), conditionally (sent only to those who complete the survey), or offered in the form of a lottery. Monetary incentives were shown to be effective in a systematic review of 94 postal surveys [22], with the odds of response being almost doubled compared to no incentive, although results were heterogenous. Monetary incentives were more effective than non-monetary incentives, and larger amounts generally more effective than smaller amounts. Unconditional incentives were more effective than conditional incentives, but there was heterogeneity among studies. In this review only one internet survey that assessed a monetary incentive versus no incentive was included, and there was no significant effect on the response rate. However six trials assessing non-monetary incentives found that the odds of response almost doubled compared to no incentive.
A variety of alternative or combined incentives have been assessed for internet surveys in terms of response rate and cost-effectiveness. For example, a Canadian study using mailed invitations for a community internet survey found a small ($2) prepaid cash incentive generated the highest response rate, but a high value lottery (2 prizes of $250) was the most cost-effective option (other options tested were no incentive and 10 prizes of $25) [27]. A systematic review of material incentives found a fairly consistent positive effect on increasing the number of people visiting the first page on internet survey websites but a more variable impact on the number of surveys actually completed [28], but most individual studies had insufficient power to detect an effect.
In 2011 we carried out a community-based survey in Melbourne, Australia examining household practices regarding greywater use and associated health risks. Greywater is used water collected from the bathroom, laundry or kitchen for subsequent reuse in the home or garden. We used postal invitations in combination with three modes of survey administration (postal, internet and telephone) and two styles of contact approach (personalised and generic), with a lottery incentive. We present a comparison of response rates and cost-effectiveness for the different modes and approaches.
