This study examined the impact of several strategies on postal survey response rate. Response quality and cost-effectiveness were also examined. Overall, a response rate of 17% was achieved. Adult participants who received the short version of the questionnaire, and those living in the relatively affluent electoral wards, were approximately 50% more likely to respond than those who received the long version of the questionnaire and those living in the relatively deprived wards respectively. Encouragingly, item non-response was relatively low in both questionnaires, but as expected was higher in the long questionnaire.
Several strategies were used to maximise the response rate, including pre-notifying participants of the survey and entering respondents into a prize draw. Nonetheless, the overall response rate was low. This response rate is comparable to that obtained in another similar postal survey , although two other recent surveys have reported response rates of 70% and 33% respectively [18, 19]. All three studies were similar to ours in that they included questions on physical activity behaviour, attitudes towards physical activity and perceptions of the neighbourhood environment. However, our questionnaire also included detailed questions on travel behaviour which required participants to recall the travel modes, durations and distances of all journeys taken. These complex but important questions may have deterred participation. A recent study conducted in Glasgow, Scotland, using comparable procedures - albeit with a more deprived population - included measures of travel and physical activity behaviour and obtained a similar response rate of 15% . The low response rate achieved in this and other studies [17, 20, 21] may also reflect a more general downward trend in participation in population surveys irrespective of the mode of data collection [4, 22, 23].
Our findings contradict previous reports that above a relatively low threshold, questionnaire length has no influence on unit non response [7, 24]. Of all the survey design factors examined in this trial, questionnaire length was the most influential. This finding may be partly explained by the length of the questionnaires tested, both of which were longer than those issued in most previous studies of the influence of length on response. A study following up mental health patients examined questionnaires of comparable length to those used in our study and found that a 13-page questionnaire elicited a greater response than a 23-page questionnaire . Questionnaires of the length used in our study are typical in the fields of travel and physical activity research, as the behaviours of interest are complex and difficult to assess with only one or two items. Our findings suggest that when developing questionnaires of this nature, there is value in reducing the length of the questionnaire as an increase in length may reduce the response rate.
In developing the short questionnaire we omitted a detailed instrument assessing travel behaviour in the previous seven days and substituted a comprehensive measure of physical activity (RPAQ) with a shorter measure with poorer validity (IPAQ). We considered that this loss in detail regarding travel and physical activity behaviour was adequately compensated for by the increased response rate achieved. Researchers should remain mindful of questionnaire length and carefully consider the trade off between the value of additional questions and the value of a larger sample.
As well as influencing unit non-response, questionnaire length also influenced item-non response which overall, was 5.8% for the short questionnaire compared with 9.8% for the long questionnaire. Although counterintuitive, our findings indicate that item non-response to the demographic questions (Section G) was higher in the short questionnaire. It could be that the when completing a longer questionnaire respondents are more likely to become desensitised to answering personal questions.
In those participants who received a reminder, personally addressing the survey pack and the subsequent reminder increased the odds of response by an estimated 44%. Given that almost 90% of the sample in this study required a reminder, personalisation may be an important strategy to apply if low response rates are expected, particularly if multiple reminders might be used (although the use of multiple reminders was not investigated in the current study). However, the effect of personalisation as a whole was a non-significant 20% increase in the odds of response, a finding which is consistent with previous research . Despite the 20% increase in response, it seems important to be aware of the self-selection biases that may apply to sampling frames such as the edited ER from which a personalised mailing list can be derived.
In this study, sending a reminder pack increased the odds of response by 60% compared with sending only a reminder postcard. In light of the low response rate, sending a reminder pack therefore appears a worthwhile strategy, although the cost per returned questionnaire was £11.8 higher than when a reminder postcard was used. Where funds are limited, a reminder postcard appears to be an efficient, albeit less effective, method of increasing participant response.
A key strength of this study was that participants were randomly allocated to study arms, which ensures high internal validity. The study was purposely conducted during September and October, a time of year when UK residents are more likely to be at home (as opposed to away on holiday) and when extreme weather patterns are unlikely to influence their travel behaviour. There was, however, a national postal strike during the period of data collection which caused considerable delays in delivery and resulted in a backlog of undelivered mail which may have undermined the response rate. The measure of PA used in the short questionnaire was different to that which was used in the long questionnaire. Subsequently the difference in unit and item non response between the two questionnaires may also have been due to the fact that different questions were included, and not solely due to the length of the questionnaires. Another limitation of the study is that the ER does not include demographic information, so the extent to which respondents were representative of those sampled could not readily be determined. Finally, to control for the possible influence of socioeconomic status on survey response we selected two relatively deprived and two relatively affluent wards, therefore the response rate achieved from sampling from wards in the middle of the socioeconomic spectrum is unknown.