Edwards et al [6] showed in their meta-analyses that the odds of response to postal questionnaires were doubled when a monetary incentive (i.e. cash) was used (odds ratio 2.02; 95% CI 1.79 to 2.27) and almost doubled when incentives were not conditional on response (1.71; 1.29 to 2.26). Contacting participants before sending questionnaires also increased response (1.54; 1.24 to 1.92), as did follow-up contact (1.44; 1.22 to 1.70) and providing non-responders with a second copy of the questionnaire (1.41; 1.02 to 1.94). Our evidence-based strategy used these methods to improve response rates and was already a comprehensive strategy before introducing the monetary incentives. The inclusion of £5 direct payment incentive unconditional on response increased absolute response rates by 10%. A similar study showed that direct payment of £5 on receipt of completed questionnaire increased response rates by 12% and significantly increased odds of response (odds ratio = 1.7, 95% CI 1.1 to 2.6, p = 0.013) [11]. An evidence-based strategy that incorporates an unconditional direct payment to patients can therefore help achieve high response rates for the completion of postal questionnaires. An explanation for this has been that an unconditional incentive promotes social exchange and a sense of reciprocal obligation. In contrast, conditional incentives may change the nature of the incentive from social to economic and so easier for respondents to decline if, for example, the amount is too low [12]. The timing of using the incentive at follow-up might also be important. We implemented the incentive at the second follow-up because we met our target at the initial follow-up. Using the incentive at initial follow-up may be even more effective at improving responses and those at subsequent follow-ups. It is possible, however, that the use of direct payments in research will raise patients' expectations of this in the future. This could affect patient's altruism and reduce the effectiveness of direct payments, or require the need for larger incentives.
There is also evidence that unconditional incentives are cost-effective in non-trial settings. The advantage depends on factors such as size of the incentive, structure of study costs and the inherent interest of the questionnaire to respondents [13]. In our trial we found that the cost per completed response in the direct payment group was higher because of the addition of the £5 incentive and the extra cost per additional respondent from using the incentive was close to £50. When considering the trial budget to recruit a target of 500 patients was £650,000 at a cost of £1300 per patient (£650,000/500) an extra fifty pounds per respondent is a minimal additional cost. With regards to the size of the incentive, £5 in our study was adequate for achieving the desired effect but we could have offered a different amount to patients to cover any expenses incurred when completing questionnaires. The payment given to patients should reflect the likely time to be spent on completing the questionnaire because if the payment provided is less (or even more) than the value that the recipient places upon their time then the incentive may not have the desired effect. Maximising response rates is important for ensuring that the study has adequate power to detect a statistically significant difference between the two treatment groups. The effect of the monetary incentive on response rates in our study helped to ensure a gain in power and precision beyond what was required for the sample size calculation. The addition of an incentive, however, might affect who responds as shown in the payment group with younger people less likely to complete questionnaires as well as males and people with lower self-reported emotional impact of knee problem.
Limitations of our study include the observational design: there was no random allocation of patients to the direct payment. The incentive was introduced for patients recruited from different practices who were asked to complete questionnaires at different times in the year. This could result in a biased estimate of the effect of the incentive. We did include important prognostic variables in our regression model to control for potential confounding factors, but our findings were obtained in patients with knee problems in primary care and may not apply to other populations.