The purpose of this study was to assess the agreement between results of telephone interviews and self-administered mailed questionnaires for the SF-12 and the GCPS, two important instruments in clinical research and practice, with the help of a two-period change-over design. To our knowledge no such data have ever been obtained for the GCPS. The study should provide insight as to whether patients' response behaviour is influenced by motivational aspects, information that would be useful for planning clinical and epidemiological trials.
The results of equivalency testing show that the response behaviour of chronic pain patients is subject to different motivational mechanisms, depending on whether the questions concern mental or physical health. Patients gave a more positive estimation of their mental health in telephone interviews than in the self-administered questionnaires. The same was not true for the SF-12 physical scales or the GCPS subscales. The most likely explanation is that the taboo that society still places on mental disability seems to cause patients speaking with another person in a telephone interview to minimise mental problems that accompany physical illness. This tendency is less likely to affect responses to the more anonymous self-administered questionnaires.
Another result of this study is that computer-assisted telephone interviews have clear advantages over mailed self-administered questionnaires when it comes to completeness of data. In addition to this previously recognized advantage of telephone interviews, however, we found that the number of missing responses was closely related to question content. There tended to be more missing responses for items concerning mental status than for those relating to physical condition. These findings support the hypothesis that patients suffering from chronic pain often view their illness as purely physical and therefore shy away from answering questions about their mental state. In the telephone interview, on the other hand, the trained interviewer is able to obtain substantially more complete responses.
A third result of this study is that, in contrast to previous findings [2, 15, 16], the level of agreement between SF-12 scores in TI and SAQ mode for the mental health subscale was dependent on the survey mode sequence. The differences were more marked if the patient had first given a more positive assessment of mental status in the telephone interview. These results can be explained with the help of findings from the psychology of memory (e.g. [17]): a more positive assessment of mental health status in the telephone interview is associated in the respondent's memory with more positive emotions, which facilitate retrospective recall of memory content when the patient subsequently fills out the paper questionnaire. A more negative assessment of physical health would have the reverse effect: negative emotions block recall of memory content, making it more likely that the respondent will describe his or her current physical state in the subsequent survey. The point in time when patients were informed about the second survey had no effect on response behaviour. Evidently the announcement to participants that they will be asked to complete a second survey is understood as information at a formal level only. Cognitive and emotional processing related to estimating one's own state of health is not likely to be influenced by when this information is received.
The strengths of our study are the comparison of GCPS values in the two survey modes, and the new control variable "Point in time when patients were informed of second survey." To our knowledge neither of these has done before.
One limitation of our study is that we cannot rule out a real remission of symptoms in the interval between the administration of the two surveys, which was at least 14 days, and therefore cannot rule out the possibility that response behaviours were influenced by real symptom improvements.
Another limitation is the non-response rate of close to 25%, as the response behaviour of this group could well differ from that of the rest of the study population. However, a systematic bias resulting from a high non-response rate is more likely to occur when the purpose of the survey is to measure treatment success, which was not the case in the present study. A more generous time frame for the return of questionnaires might increase the response rate. In this study, 662 patients did not return the questionnaire until after the four-week period allotted. The average time taken by those patients to return the questionnaire was seven weeks.
There are also some practical disadvantages to using the CATI-System for data collection: depending on the population size and the technical equipment available, this mode is more time consuming and more costly than mailing out paper questionnaires. Administration of surveys via the Internet might be a cost-effective alternative. A factor to be considered, however, is that proportionately fewer people have access to this means of communication than to the telephone.
Chronic pain research and therapy has traditionally been an interdisciplinary undertaking. Besides quality of life questionnaires, other important data gathering instruments are questionnaires that measure fear or depression. Using instruments such as the CES-D in telephone interviews[18] could, by analogy to results obtained for the mental health subscale of the SF-12 in our study, lead to a systematic underestimation of depression.
Since telephone interviews offer significant advantages over self-administered questionnaires, further mode-comparing studies in this area, particularly with chronic pain patients, are clearly needed.