- Research article
- Open Access
- Open Peer Review
Effect of questionnaire structure on recall of drug utilization in a population of university students
© Gama et al; licensee BioMed Central Ltd. 2009
- Received: 08 February 2009
- Accepted: 29 June 2009
- Published: 29 June 2009
Self-reported data are a common source of information about drug exposure. Modes of data collection differ considerably and the questionnaire's structure may affect prevalence estimates. We compared the recall of medication use evaluated by means of two questionnaires differing in structure and length.
Drug utilization was assessed by two alternative versions of a questionnaire (A – 4 pages, including specific questions for 12 indications/pharmacological groups and one question for "other medicines"; B – 1 page, including 1 open-ended question to cover overall drug consumption). Each of 32 classes in a private University in Maputo, Mozambique, was randomly assigned questionnaire A (233 participants) or B (276 participants). Logistic regression (allowing for clustering by classroom) was used to compare the two groups in terms of socio-demographic characteristics and medication used during the previous month.
Overall, 67.4% of the subjects had used at least one drug during the previous month. The following prevalences were greater among participants completing questionnaire A: use of drugs from two or more pharmacological groups (60.5% vs. 34.4%, p < 0.001), use of two or more drugs (66.2% vs. 43.0%, p < 0.001), and use of antibiotics (14.6% vs. 6.9%, p = 0.001), antifungals (9.4% vs. 4.0%, p = 0.013), antiparasitics (5.6% vs. 1.8%, p = 0.031) and antacids (8.6% vs. 3.6%, p = 0.024). Information about duration of treatment and medical advice was more complete with version A.
The indication/drug-specific questions (questionnaire A) revealed a significantly higher prevalence of use of medicines – antibiotics, antifungals, antiparasitics and antacids – without compromising the completeness of the information.
- Medical Advice
- Drug Utilization
- Questionnaire Design
- Separate Question
- Anatomical Therapeutic Chemical Classification
Modes of data collection by questionnaire may differ in several ways, including the methods for contacting respondents and for questionnaire delivery, and the administration of questions (e.g. the appearance of the questionnaire, the language used and the cultural adaptation and wording). These differences may affect the reliability and validity of the method [6–8].
Regarding drug utilization, questionnaire characteristics such as question wording and question or response order or format can be variously associated with exposure misclassification. Studies addressing the effect of questionnaire design on the recall of pharmacological treatments are heterogeneous in aim, method and quality, but are unanimous in concluding that indication- or drug-oriented questions reveal higher prevalences of drug utilization than open-ended questions . However, obtaining data related to various drug categories frequently requires lengthy questionnaires including separate questions for each of the main pharmaceutical groups under study. It may not be possible or desirable to use large number of questions for this purpose, as that may substantially increase the size of the questionnaire and lead to poor acceptability and participant compliance. A short questionnaire may have the advantages of lower respondent burden and lower administrative costs, but its ability to characterize drug utilization fully is expected to be more limited. Shorter questionnaire versions can hardly include specific questions about the different pharmacological groups being assessed, which may compromise recall, and the potential for a lower burden may be attenuated by the use of more complex questions or a less attractive appearance, with many questions concentrated in a small area [10, 11].
We hypothesized that a single-item questionnaire complemented with examples of the indications and medicines that are expected to be more frequent in the population studied could yield similar estimates of the prevalence of drug utilization to an extended multi-item questionnaire addressing the utilization of medicines for several indications. Therefore, our aim was to compare recall using two questionnaires differing in structure and length (long questionnaire including indication/drug-specific questions vs. short questionnaire including one open-ended question and several examples of indications and drugs).
Two alternative versions (A and B) of the questionnaire, differing in question structure and length, were used to collect information. Both versions started with the same close-ended question "Did you use any medication in the last month (including tablets, capsules, injections, ointments, ovules, syrups, etc.)?" and gave similar general instructions to the participants. The questionnaires answered by the participants (versions A and B) were in Portuguese. English versions are provided as additional files 1 and 2.
Indication/Pharmacological groups and examples presented in both questionnaire versions, and subsequent classification according ATC nomenclature.
Classification (level II ATC subgroups)
Medications for treatment of pain and inflammation
Paracetamol, Voltaren®/diclofenac, ibuprofen, etc.
Analgesics (M01: Anti-inflammatory and anti-rheumatic drugs, non steroids, N02: Analgesics, B01: Antithrombotic agents)
Medications for treatment of flu or cold
Cêgripe®, Constipal®, Corenza® C
amoxicillin, tetracycline, co-trimoxazol, metronidazol, etc.
Antibiotics (J01: Antibacterials for systemic use)
Antimicotics/antifungals for treatment of infections
Canesten®, Clotrimazol®, Quadriderme®, Nalbix®, etc
Antifungals (ATC D01: Antifungals for dermatological use; G01: Gynecological antiinfectives and antiseptics; D07: Corticosteroids, dermatological preparations, combinations with antibiotics, J02: Antimicotics for systemic use)
artemisine + fansidar®, etc
Antimalarials (ATC P01B: Antimalarials)
albendazol, mebendazol, etc
Antiparasitics (P01, except P01B: Antiprotozoals; P02: Antihelmintics)
Vitamins and minerals
multivitamins, complex B, ferrous salt, vitamin C, etc.
Vitamins (ATC A11: Vitamins, B03: Antianemic preparations)
salbutamol/Ventilan®, aminofiline, beclomethasone, prednisolone, etc.
Antiasthmatics (R03: Drugs for obstructive airway diseases, H02: CCT for systemic use
clorfeniramine, loratadine, Claritine®, etc.),
Anti-histamines (ATC R06: Antihistamines for systemic use)
Diane® 35, Microginon®, etc.
Hormonal contraceptives for systemic use (ATC G03: Sex hormones and modulators of the genital system; L02: Endocrine Therapy)
Antitussives and/or expectorants
Benilyn®, Diacol®, Benetussin®, Tosseque®, sodium benzoate, etc.
Antitussives (ATC R05: Cough and cold preparations)
Medications for gastric problems
omeprazole, cimetidine, ranitidine, ENO®- fruit salts, aluminium hydroxide, Rennie®, Kompensan®, etc.
Antacids (ATC A02: Drugs for acid related disorders)
including all the drugs not included in the previously-described groups
Questionnaire B (see additional file 2: questionnaire B) was one page long and included a single open-ended question to cover overall drug consumption, preceded by a short passage of text including the same examples provided in each of the questions from questionnaire A. Subjects were asked to complete a table similar to the ones described for questionnaire A.
Each medicine mentioned in questionnaires A or B was coded to the second level of classification (therapeutic subgroup) of the WHO Anatomical Therapeutic Chemical classification (ATC) . The pharmacological groups/indications for which there were specific questions in questionnaire A and the corresponding ATC subgroups are shown in table 1. For data analysis, the medicines classified as "medications for treatment of pain and inflammation" or as "medications for treatment of flu or cold" were considered jointly as "analgesics".
Information about medical advice (to assess the use of medication following recent or past medical advice) and duration of treatment was analysed separately for drugs from each pharmacological group. For subjects using more than one medicine in the same group, the mean duration of treatment was used for analysis. For each pharmacological group, the results regarding medical advice are presented as the percentage of subjects who had used at least one medicine under a physician's advice.
To test the hypothesis that the two differently-structured questionnaires yielded the same prevalence of drug utilization, one version was randomly assigned to each of the 32 diurnal classes using a list of random numbers generated with EPi Info software, version 6.04d . Questionnaires A and B were designated to 14 and 18 classes, respectively. All subjects within the same classroom received the same questionnaire version (A: 233 subjects; B: 276 subjects).
The choice of a one-month recall period was based on the expected frequency of drug utilization in this population, which was informed by a previous survey in the same setting . This sample allows differences in the estimates of the two questionnaires to be detected with a power of 80%, for a 95% confidence level, when the magnitude of the difference corresponds to a risk ratio of two, and the proportion of drug use is above 10%, considering a design effect as high as 1.15 (which would be obtained for an average cluster size of 15 and an intra-cluster correlation coefficient of 0.01) . The assumptions about the expected design effect and average cluster size are supported by a previous similar study in the same setting .
Logistic regression with robust standard errors (allowing for clustering by classroom) was used to compare the two groups (questionnaire A and questionnaire B) in terms of socio-demographic characteristics and medication used during the previous month . All the tables present p-values corresponding to comparisons with no adjustment of covariates. The groups of medicines are presented in the tables in the order in which they appeared in Questionnaire A. Data were analyzed using STATA®, version 9.2.
The National Ethics Committee of Mozambique approved the survey protocol. Students were asked to read an informed consent form stating the general objectives of the study and data collection methods, in agreement with the Declaration of Helsinki. Only students who signed the informed consent form were allowed to participate and were asked to complete the questionnaire.
Characteristics of participants evaluated using questionnaires A (14 classrooms, 233 subjects) and B (18 classrooms, 276 subjects).
Number of subjects*
Age (% above 20)
Sex (% male)
Study area (% health related)
Grade (% above 2nd grade)
Users of at least one medicine during the previous month
The median number of different pharmacological groups used by each subject (2 vs. 1), and the percentage of participants reporting the use of drugs from two or more pharmacological groups, were higher among those completing questionnaire A (60.5% vs. 34.4%, p < 0.001). Also, questionnaire A revealed a greater median number of medicines used per person among drug users (2 vs. 1) and a greater percentage of participants reporting the use of two or more drugs (66.2% vs. 43.0%, p < 0.001). However, the median number of drugs reported for all groups was 1 in both questionnaires, except for antimalarials (A vs. B: 2 vs.1).
Medication used during the previous month, according to the type of questionnaire (n = 509).
Medication used by women and men during the previous month, according to the type of questionnaire.
Women (n = 313)
Men (n = 184)
Information about the duration of treatment with medication used during the previous month, according to the type of questionnaire.
Missing information regarding
duration of treatment, n (%)
of treatment (days)
Information about the medicines used under medical advice during the previous month, according to the type of questionnaire.
Missing information regarding
medical advice, n (%)
Subjects who had used at least one medicine under medical advice, n (%)
The longer questionnaire version, which included indication/drug-specific questions, revealed a significantly higher prevalence of use of antibiotics, antifungals/antimicotics, antiparasitics and antacids. For the remaining drug groups, recall did not differ significantly between questionnaire versions. Participants receiving the questionnaire version relying on a single open-ended question tended to provide less complete information about treatment duration and medical advice.
The differences in the estimates obtained with the two questionnaire versions were not the same across all pharmacological groups or drugs. This is consistent with previous finding in studies addressing the effect of questionnaire design on the recall of pharmacological treatments [16, 18, 19].
Ademi et al.  tested different question formats (frequency measure; symptom-oriented question; open-ended question), in a substantially different population, to quantify the prevalence of use of analgesics during the preceding seven days. The higher prevalence was obtained by asking the respondents to name the medicines they had used during the previous week, but the open-ended item was the last of three questions asked sequentially.
For antibiotics and antifungals/antimicotics the estimated prevalence differed significantly between the two questionnaire structures. This was also noted by Neutel et al.  who observed that antibiotics were better recalled when asked about by specific name rather than by drug category. Our study revealed no differences between the two questionnaire versions in assessing antimalarials, which may be because malaria is a very frequent severe disease in this setting, so we would not expect the participants to forget malaria treatments used within one month of the interview.
For pharmacological groups such as the antiasthmatics, no differences between questionnaire versions were observed in the prevalence of use, perhaps because these drugs are used to treat a chronic condition, unlikely to be forgotten by the participants, and recalled regardless of the questionnaire format used. For vitamins or antitussives, the differences observed were not significant but were consistent in men and women, suggesting that the recall may again be affected by the differences in questionnaire design.
The population evaluated in our study was relatively homogeneous in age and education and does not adequately represent Mozambican university students, still less the general population. Studies assessing the influence of education on recall in drug utilization have concluded that agreement between questionnaires and information retrieved from record data increases with education [21, 22], although less educated subjects tend to use medicines more than better educated ones . We may expect even greater differences between the results obtained with versions A and B of the questionnaire among older or less well-educated participants. The results from surveys based on self-reporting of drug intake among such populations need an even more cautious interpretation.
The internal validity of the present study is not compromised by our option for a one-month recall period, since it applied to both the questionnaire versions compared. Choosing a shorter recall period (e.g. two days or a week) would have contributed to more accurate recall of the medicines to which the participants had been exposed, but also to a lower prevalence of use. Conversely, a longer recall period (e.g. six months or one year) would have yielded worse recall  but a higher proportion of subjects using medicines. Regarding external validity, we may expect a structured questionnaire with specific questions about indication/pharmacological group to have more impact on the recall of medicines used for acute conditions, but a negligible impact when recall is only required over shorter periods.
Both groups (A and B) answered the questionnaire at the same time, so seasonal fluctuations on the intake of medicines for flu symptoms or malaria cannot be expected to have compromised internal validity. It may not be possible to generalize our results to studies conducted at other times of the year since the differences between questionnaire's versions may be related to the frequency of consumption of medicines used for seasonally variable conditions. We think it is unlikely that the recall of malaria treatment would be substantially influenced by questionnaire design, given the severity of the disease. For the remaining pharmacological groups used for acute conditions, our results do not suggest differences dependent on the frequency of consumption, beyond random variation. Also, the enhanced recall of drug utilization obtained using indication- and drug-oriented questions seems to be independent of the frequency of drug use .
The specificity of the questions about pharmacological treatments is known to contribute to more accurate estimates of medicine use . However, since it is not feasible to write all possible examples of medicines for each indication/pharmacological group, recall may differ between the examples provided and medicines that are not listed. We tried to overcome this limitation by asking simultaneously about the use of medicines for specific indications/pharmacological groups and providing examples of the medicines most commonly used in this population. Moreover, the same sets of examples were used in both versions to ensure comparability in this aspect of questionnaire design.
Since the same medicine can be used for different purposes, respondents may become confused about how to respond when indications and example medicines are provided side by side. This is particularly important for drugs such as paracetamol, which may be used both for "pain and inflammation" and for symptomatic "treatment of flu or cold". We opted to use two separate questions for these indications, expecting to improve recall, and asked participants for the intended utilization of each medicine, allowing them to be classified correctly in this regard. In addition, although we had two separate questions for medicines used for "pain and inflammation" and "flu or cold", we opted to analyze these groups together, overcoming the potential misclassification.
No records were available to check the reported drug utilization against the prescribed and non-prescribed drug data, so no inferences could be drawn about the absolute validity of exposure recall using questionnaires A or B. However, studies comparing questionnaires with pharmacy data as a gold standard have shown that drug-oriented questionnaires yield better agreement than data collected by open-ended questions [24, 21], and the design used for questionnaire A is expected to provide more accurate estimates of drug utilization [18, 21]. Also, several studies have indicated that in drug utilization studies the prevalence estimates obtained from questionnaire surveys are more reliable than information retrieved from medical or pharmacy records, especially for assessing over-the-counter drugs and medication misuse [1, 22, 25–27]. Self-medication reports may be a useful instrument, together with medical prescriptions and pharmacy records, for measuring the efficacy of the drugs used .
A concern when using a long questionnaire such as version A is the burden that it imposes to the participant, requiring a long time to answer the questions, which may result in a larger proportion of refusals and in missing data among respondents . On the other hand, the physical layout of a questionnaire is important for increasing response rates as it is sometimes easier to complete a questionnaire that, despite being longer, is clearer in structure . In our study, the time taken by the participants to complete each questionnaire version could not be quantified, but the longer version is not necessarily expected to take more time to complete as it does not differ from the shorter one in the level of detail required. Also, the indication-oriented questions in version A may have been perceived as easier to answer because they were more structured, facilitating recall of previous treatments. The proportion of refusals was low and similar for both questionnaire versions (1.7% for version A and 2.1% for version B), probably because the questionnaires were completed in the classroom. Therefore, in the present study, missing information may be a better indicator of the acceptability of the questionnaire than the proportion of refusals.
The proportion of subjects with missing information about the duration of treatment or medical advice was even lower among those receiving the 4-page questionnaire, suggesting that both versions were well accepted by the participants and that the layout of the longer questionnaire facilitated more complete recall.
We found that a substantial proportion of students used medicines without medical advice, which would not be identified in a study based on doctor-generated or pharmacy-based prescription records. This finding has implications for studies relying on such databases and raises the concern that such information includes false positive «exposures» (prescriptions issued or completed but not consumed).
In self-administered questionnaires, the first options in a list of answering options are more likely to be chosen (response choice order effects ). We could eventually expect a similar phenomenon in the examples provided in questionnaire B, or question order effects in questionnaire A. The former would lead to larger differences between questionnaire versions, the latter to smaller differences. However, no consistent pattern was observed across the last groups provided as examples (version B) or specifically asked (version A), and the different characteristics of the last indications/pharmacological groups preclude such an interpretation of the findings.
Our results add to previous research by demonstrating that questionnaire structure and layout contribute to differences in recall of pharmacological treatments, even when the content of the questionnaire and the type of questions are virtually the same. We also demonstrated that a larger questionnaire does not necessarily lead to less complete recall.
Future research on this topic should assess the impact of data collection instruments across populations with different patterns of medicine utilization and ability to recall usage over different periods.
For a proper interpretation of data from drug utilization studies it is essential to take into account the characteristics of the instruments used for data collection, and their potential impact on the validity of the estimates for different pharmacological groups and across settings.
The authors acknowledge the contributions of Artur Fernandes and Neila Martins to the data collection.
- Al-Windi A: The validity of a questionnaire on medicines used in health care practice: comparison of a questionnaire and computerized medical record survey. Eur J Clin Pharmacol. 2003, 59: 321-329. 10.1007/s00228-003-0615-1.View ArticlePubMedGoogle Scholar
- Bergman U, Andersen M, Vaccheri A, Bjerrum L, Wettermark B, Montanaro N: Deviations from evidence-based prescribing of non-steroidal anti-inflammatory drugs in three European regions. Eur J Clin Pharmacol. 2000, 56: 269-272. 10.1007/s002280000144.View ArticlePubMedGoogle Scholar
- Campbell SM, Cantrill JA, Roberts D: Prescribing indicators for UK general practice: Delphi consultation study. Bmj. 2000, 321: 425-428. 10.1136/bmj.321.7258.425.View ArticlePubMedPubMed CentralGoogle Scholar
- Cars O, Molstad S, Melander A: Variation in antibiotic use in the European Union. Lancet. 2001, 357: 1851-1853. 10.1016/S0140-6736(00)04972-2.View ArticlePubMedGoogle Scholar
- Lee D, Bergman U: Studies of Drug Utilization. Pharmacoepidemiology. Edited by: Strom BL. 2005, West Sussex: Wiley, FourthGoogle Scholar
- Bowling A: Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf). 2005, 27: 281-291. 10.1093/pubmed/fdi031.View ArticleGoogle Scholar
- Choi BC, Pak AW: A catalog of biases in questionnaires. Prev Chronic Dis. 2005, 2: A13-PubMedGoogle Scholar
- Tolonen H: Towards the High Quality of Population Health Surveys. Standardization and quality control. Academic dissertation. 2005, National Public Health Institute Helsinki, Finland and Faculty of Medicine, University of Kuopio, FinlandGoogle Scholar
- Gama H, Correia S, Lunet N: Questionnaire design and the recall of pharmacological treatments: a systematic review. Pharmacoepidemiol Drug Saf. 2009, 18: 175-187. 10.1002/pds.1703.View ArticlePubMedGoogle Scholar
- Hing E, Schappert SM, Burt CW, Shimizu IM: Effects of form length and item format on response patterns and estimates of physician office and hospital outpatient department visits. National Ambulatory Medical Care Survey and National Hospital Ambulatory Medical Care Survey, 2001. Vital Health Stat. 2005, 2: 1-32.Google Scholar
- Bowling A: Just one question: If one question works, why ask several?. J Epidemiol Community Health. 2005, 59: 342-345. 10.1136/jech.2004.021204.View ArticlePubMedPubMed CentralGoogle Scholar
- Lucas R, Lunet N, Carvalho R, Langa J, Muanantatha M, Nkunda LP, Barros H: Patterns in the use of medicines by university students in Maputo, Mozambique. Cad Saude Publica. 2007, 23: 2845-2852. 10.1590/S0102-311X2007001200005.View ArticlePubMedGoogle Scholar
- WHO: Guidelines for ATC classification. 1990, Oslo: WHO Collaborating centre for drug statistics methodology, Nordic Council on medicinesGoogle Scholar
- Dean AGDJ, Coulombier D, Burton AH, Brendel KA, Smith DC: Epi Info, version 6.04d: a word processing database and statistics for epidemiology in microcomputers. 1996, Atlanta, USA: Centers for Disease Control and Prevention, Geneva, Switzerland: World Health OrganizationGoogle Scholar
- Bland JM: Cluster randomised trials in the medical literature: two bibliometric surveys. BMC Med Res Methodol. 2004, 4: 21-10.1186/1471-2288-4-21.View ArticlePubMedPubMed CentralGoogle Scholar
- Lunet N, Bastos J, Cumaio F, Silva P, Dias E, Barros H: Recall of drug utilization depends on subtle structural questionnaire characteristics. Pharm World Sci. 2008, 30: 175-181. 10.1007/s11096-007-9161-8.View ArticlePubMedGoogle Scholar
- Peters TJ, Richards SH, Bankhead CR, Ades AE, Sterne JA: Comparison of methods for analysing cluster randomized trials: an example involving a factorial design. Int J Epidemiol. 2003, 32: 840-846. 10.1093/ije/dyg228.View ArticlePubMedGoogle Scholar
- Mitchell AA, Cottler LB, Shapiro S: Effect of questionnaire design on recall of drug exposure in pregnancy. Am J Epidemiol. 1986, 123: 670-676.PubMedGoogle Scholar
- Neutel CI, Walop W: Comparing two different approaches to measuring drug use within the same survey. Chronic Dis Can. 2000, 21: 150-156.PubMedGoogle Scholar
- Ademi Z, Turunen JH, Kauhanen J, Enlund H: A comparison of three questionnaire-based measures of analgesic use over 11 years in adult males: a retrospective analysis of data from a prospective, population-based cohort study. Clin Ther. 2007, 29: 529-534. 10.1016/S0149-2918(07)80090-7.View ArticlePubMedGoogle Scholar
- Klungel OH, de Boer A, Paes AH, Herings RM, Seidell JC, Bakker A: Influence of question structure on the recall of self-reported drug use. J Clin Epidemiol. 2000, 53: 273-277. 10.1016/S0895-4356(99)00167-5.View ArticlePubMedGoogle Scholar
- Goodman MT, Nomura AM, Wilkens LR, Kolonel LN: Agreement between interview information and physician records on history of menopausal estrogen use. Am J Epidemiol. 1990, 131: 815-825.PubMedGoogle Scholar
- WHO: EUROHIS: Developing common instruments for health surveys. Edited by: Nosikov A, Gudex C. 2003, World Health Organization, Regional Office for EuropeGoogle Scholar
- Berg de Jong-van den LT, Waardenburg CM, Haaijer-Ruskamp FM, Dukes MN, Wesseling H: Drug use in pregnancy: a comparative appraisal of data collecting methods. Eur J Clin Pharmacol. 1993, 45: 9-14. 10.1007/BF00315343.View ArticleGoogle Scholar
- McCabe SE: Correlates of nonmedical use of prescription benzodiazepine anxiolytics: results from a national survey of U.S. college students. Drug Alcohol Depend. 2005, 79: 53-62. 10.1016/j.drugalcdep.2004.12.006.View ArticlePubMedPubMed CentralGoogle Scholar
- Galduroz JC, Noto AR, Nappo SA, Carlini EA: Trends in drug use among students in Brazil: analysis of four surveys in 1989, 1993 and 1997. Braz J Med Biol Res. 1987, 37: 523-531.View ArticleGoogle Scholar
- West SL, Savitz DA, Koch G, Strom BL, Guess HA, Hartzema A: Recall accuracy for prescription medications: self-report compared with database information. Am J Epidemiol. 1995, 142: 1103-1112.PubMedGoogle Scholar
- Laporte JR, Porta M, Capella D: Drug utilization studies: a tool for determining the effectiveness of drug use. Br J Clin Pharmacol. 1983, 16: 301-304.View ArticlePubMedPubMed CentralGoogle Scholar
- Boynton PM, Greenhalgh T: Selecting, designing, and developing your questionnaire. Bmj. 2004, 328: 1312-1315. 10.1136/bmj.328.7451.1312.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/9/45/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.