Towards socially inclusive research: An evaluation of telephone questionnaire administration in a multilingual population

Missing data may bias the results of clinical trials and other studies, with low response rates compromising the validity of the findings [1, 2]. Acceptable questionnaire response rates are considered to be in the range of 60–70%, with response rates of over 70% described as very good [3]. Self administered questionnaires sent and returned by post offer a cost effective way to obtain data from a large number of participants, but often has low response rates. A recent systematic review of methods to increase response rates to questionnaires identified a number of strategies such as shortening questionnaires, repeat mailing of questionnaires and telephone reminders [4]. Intensive reminders by telephone and post were the most effective, improving response rates by an average of 24%. Financial incentives have also been used to increase response rates [5]. For example a recent descriptive study reported that an unconditional payment of £5 increased the response rate from 78 to 88% [6].

Some studies comparing telephone and postal response rates from patients have found that response rates from questionnaires administered by telephone are higher [7] while others have found no difference [8]. Studies comparing responses from general practitioners have also shown varying results. One study reported a lower response rate for postal surveys than for telephone methods [9], while another found a higher response rate for postal surveys than for telephone methods [10]. Hocking and colleagues suggested that the lower telephone response rate in their study was because practice receptionists blocked telephone access to the general practioners who were the target of the survey. A consensus is emerging that a combination of direct contact and postal methods leads to higher response rates to questionnaires than the use of postal methods alone [8, 11].

Two areas of uncertainty exist about the use of telephone compared with postal administration of questionnaires. First, research studies not only need a good response rate, but also representative responses from different socio-economic, demographic and clinical groups. It is not known whether postal and telephone methods of administering questionnaires affect responses differently from different socio-economic, demographic or clinical groups. Second, responses to questionnaire items administered by post or telephone need to be equivalent. Only two randomised studies have examined this. One found a difference in responses to the same question, while the other did not [8, 12].

The observational study described here took place within SHIFT (Screening for Haemoglobinopathies in the First. Trimester), a cluster randomised controlled trial assessing the feasibility, effectiveness and acceptability of offering antenatal Sickle Cell and Thalassaemia (SCT) screening in primary care (ISRCTN00677850) [13]. The trial was set in two inner city UK Primary Care Trusts (PCTs) with a large number of people from minority ethnic groups and with high levels of material and social deprivation. The questionnaires had been designed to assess informed choice about antenatal SCT screening in a population with low levels of literacy [14].

At the start of the SHIFT trial, participants were posted questionnaires (in one of twelve languages, selected according to information provided at trial consent) to complete at home and return by post. Initial monitoring indicated that this method gave an unacceptably low response rate. Based on a literature search we developed a second strategy that was more likely to achieve a high response rate in a multilingual population namely, offering participants the opportunity to complete a questionnaire over the telephone with a researcher and an interpreter, if necessary. Participants offered the second strategy were also offered postal completion. This paper compares (i) the response rates of offering participants a choice of telephone and postal completion methods with those of offering postal completion only; (ii) the responses obtained from the method selected ie postal or telephone completion; (iii) the financial costs to the research team of offering postal and telephone administration of questionnaires in order that reliable estimates can be included in future research cost estimates.

More women were identified as having positive attitudes towards undergoing antenatal SCT screening using telephone questionnaires than using postal questionnaires (96% vs 87%, 95% CI Difference 0.006 to 15%). There were no differences in the proportion of women identified with good knowledge about the test using telephone or postal questionnaires ( 37% vs 31% 95% CI diff -6 to 17% ) (Figure 1)

Telephone questionnaires were completed in twenty languages other than English. Postal questionnaires were completed in nine languages other than English.

The results of this study suggest that offering a choice of telephone or postal completion methods can result in a dramatic increase in response rates compared with postal completion alone. This study was not a randomised trial and there could be other explanations for the observed effect. The intervention was complex. It included offering a choice, and offering a choice of two methods that differed in several ways. While we cannot exclude the possibility that the mere offer of a choice was responsible for the effect, the pattern of results suggests that it is the opportunity to complete questionnaires by telephone that is crucial. Telephone completion compared with postal completion allows for ready translation to more languages and provides social support to complete the questionnaire. Perhaps most importantly it removes the reading obstacle to questionnaire completion thereby allowing the estimated 20–25% of the UK population who are functionally illiterate [20] to participate in the research process.

The increased response rate obtained by administering questionnaires by telephone raises two questions: first, did the increase in response rate vary by social group; and second, were responses obtained by telephone different to those obtained by post. Regarding differential response rates across social groups, the response from those offered a choice in this study was higher for English speakers than for non English speakers. The lack of data from non responders means, however, that we are not able to assess how representative the responders are of the telephone or postal groups overall.

The data do not identify a reason why non-English speakers were less likely to opt for completing the questionnaire by telephone than were English speakers. It may reflect a failure of the trial to engage non-English speakers in the research process. Alternatively, it may reflect cultural differences in, for example, willingness to talk to strangers over the telephone. Further work is required to understand why non-English speaking women were less likely to choose to complete questionnaires by telephone than were English speaking women. Understanding this may allow telephone administration to be offered in ways that increase acceptability, and hence, use for non-English speakers and thereby increase response rates further.

Comparing responses from the two modalities revealed differences in assessed attitudes: women completing the questionnaire over the telephone were more likely to be classified as having positive attitudes towards undergoing antenatal SCT screening than women completing the postal questionnaire. There were no differences in knowledge by completion modality. One explanation is that the difference may be due to the way the questions were asked or social desirability. Alternatively it may be because the study was not randomised. That is women who opted to complete the questionnaire over the telephone had more positive attitudes towards undergoing the test than women who completed the questionnaire by post.

There are financial costs involved in offering women a choice of telephone or postal questionnaire completion methods. These include the researcher time to administer the questionnaire over the telephone as well as the cost of using telephone interpreters. The use of telephone interpreters did not negate the need (or cost) for translating written materials. Research budgets should include funding to cover all these costs.

There are potential problems associated with administering questionnaires over the telephone. The person administering the questionnaire needs to be trained to ensure that the potential participant does not feel under any pressure to participate in the evaluation or to complete the questionnaire. Training is also required to ensure that questions are not asked in a leading way i.e. in a way likely to guide respondents to answer in a particular way.

For women who opted to complete the questionnaire by telephone, the use of interpreters did not appear to pose any problems. The use of telephone interpreters allows greater flexibility than using written translations because the languages required do not require specifying in advance. However the use of telephone interpreters does not allow for the use of quality control procedures such as back translation that are available with written translations.

There has been some debate about the use of translation services within the NHS which cost in the region of £55 million per annum [21]. It has been argued that the use of such services may compound rather than ameliorate the health problems of non-English speakers by reducing the need for them to learn English and thereby implicitly encouraging non-English speakers to remain outside of the dominant culture [22]. Others have argued that the lack of translation services results in poorer healthcare for non-English speakers [23]. Whilst this debate is likely to continue within the context of service provision, it is important to acknowledge that this debate is not applicable in research settings. A prerequisite for reliable trials is that trial outcomes are obtained for all participants and are representative of the population in general [2]. They therefore need to include people who do as well as those who do not speak English.

Studies requiring data to be collected by questionnaire may obtain higher response rates from both English and non-English speakers when a choice of telephone or postal administration and where necessary, an interpreter, is offered compared to offering postal administration only. This approach will, however, incur additional research costs and uncertainty remains about the equivalence of responses obtained in the two methods.