- Research article
- Open Access
Validation of the diabetes, hypertension and hyperlipidemia (DHL) knowledge instrument in Malaysia
BMC Medical Research Methodology volume 12, Article number: 18 (2012)
Patient's knowledge on diabetes, hypertension and hyperlipidaemia and its medications can be used as one of the outcome measures to assess the effectiveness of educational intervention. To date, no such instrument has been validated in Malaysia. Therefore, the aim of this study was to evaluate the validity and reliability of the Diabetes, Hypertension and Hyperlipidemia (DHL) knowledge instrument for assessing the knowledge of patients with type 2 diabetes in Malaysia.
A 28-item instrument which comprised of 5 domains: diabetes, hypertension, hyperlipidemia, medications and general issues was designed and tested. One point was given for every correct answer, whilst zero was given for incorrect answers. Scores ranged from 0 to 28, which were then converted into percentage. This was administered to 77 patients with type 2 diabetes in a tertiary hospital, who were on medication(s) for diabetes and who could understand English (patient group), and to 40 pharmacists (professional group). The DHL knowledge instrument was administered again to the patient group after one month. Excluded were patients less than 18 years old.
Flesch reading ease was 60, which is satisfactory, while the mean difficulty factor(SD) was 0.74(0.21), indicating that DHL knowledge instrument was moderately easy. Internal consistency of the instrument was good, with Cronbach's α = 0.791. The test-retest scores showed no significant difference for 26 out of the 28 items, indicating that the questionnaire has achieved stable reliability. The overall mean(SD) knowledge scores was significantly different between the patient and professional groups [74.35(14.88) versus 93.84(6.47), p < 0.001]. This means that the DHL knowledge instrument could differentiate the knowledge levels of participants. The DHL knowledge instrument shows similar psychometric properties as other validated questionnaires.
The DHL knowledge instrument shows good promise to be adopted as an instrument for assessing diabetic patients' knowledge concerning their disease conditions and medications in Malaysia.
Diabetes and its complications pose a major health-care burden worldwide and present major challenges to patients, health-care systems, and national economies. The World Health Organization estimates that between 2000 and 2030, the world population will increase by 37% and the number of people with diabetes will increase by 114% . Asia will be the major site of a rapidly emerging diabetes epidemic based on population growth, an increase in the elderly population (> 65 years of age), and the rate of urbanisation . India and China will remain the two countries with the highest numbers of people with diabetes (79·4 million and 42·3 million, respectively) by 2030 . In Malaysia, the prevalence of diabetes has escalated from 6.3% in 1986, to 8.3% in 1996, and to 14.9% in 2006 . One in six Malaysians above the age of 30 years old has diabetes . The dramatic increase in the prevalence of diabetes may be due primarily to the rise in obesity in Malaysia during the past decade, which has been found to be associated with rapid urbanization [2, 3].
Research has shown that the provision of diabetes education by healthcare professionals on medications, diet, exercise, home glucose monitoring, foot care, and treatment modifications have improved clinical outcomes and the quality of life of patients [4–6]. Although knowledge alone does not guarantee a change in behaviour or lead to effective self-management , the assessment of diabetes-related knowledge is an important step towards providing individualize diabetes education programs and to evaluate the effectiveness of such interventions .
A literature search revealed that several instruments for assessing diabetes knowledge have been developed: the Michigan Diabetes Knowledge Tool (MDKT) , the Diabetic Knowledge Questionnaire (DKQ) [10, 11], the Diabetes Knowledge Assessment (DKN) scale , the Revised Diabetes Knowledge Scale , the PedCarbQuiz (PCQ)  and the Diabetic Numeracy Test (DNT) . Most of these instruments were developed and validated in the United States [9–11, 14, 15], whilst others were developed and validated in Australia  and the United Kingdom . The earliest instrument, the MDKT was developed and validated in the 1970s . Most of these instruments were developed in the English Language [9–11, 14, 15]. However, several instruments have since been translated to other languages such as Spanish [10, 11], Portuguese  and Bahasa Malaysia [18, 19].
These instruments varied widely in their assessment objectives. The MDKT assessed knowledge on general issues and insulin use , the DKQ assessed general diabetes knowledge [10, 11], the PCQ assessed carbohydrate food recognition, carbohydrate food counting and the incorporation of carbohydrate counting in calculating insulin dose , whilst the DNT measured numeracy skills for diabetes (food label interpretation, calculation of insulin dose based on blood, glucose and carbohydrate corrections) .
The population in which these instruments were assessed varied between studies. Instruments were validated in adults with type 1 or type 2 diabetes [9, 15], in adults with type 2 diabetes only [10, 11], or in adolescents or children with type 1 diabetes [14, 15].
Patient's knowledge on diabetes, hypertension and hyperlipidaemia and its medications can be used as one of the outcome measures to assess the effectiveness of educational intervention. In Malaysia, two instruments: the Orang Asli-Diabetes Knowledge Questionnaire (OA-DKQ)  and the MDKT  have been translated into Bahasa Malaysia and validated. The OA-DKQ assesses diabetes-related knowledge whilst the MDKT assesses general diabetes knowledge and insulin use. However to date, no instrument that assesses knowledge on other cardiovascular metabolic risk factors such as hypertension and hyperlipidaemia, in addition to diabetes knowledge has been developed and validated in Malaysia. Therefore, the aim of this study was to develop and validate a comprehensive questionnaire to assess patient's knowledge on diabetes, medications, and other cardiovascular metabolic risk factors which included hypertension and hyperlipidaemia, for use by Malaysians with type 2 diabetes mellitus.
Development of the Diabetes, Hypertension and Hyperlipidemia (DHL) knowledge instrument
The DHL knowledge instrument was developed for the community-based Cardiovascular Risk Factors Intervention Strategies (CORFIS) trial. The aim of this trial was to assess the efficacy of a chronic disease management strategy in the management of patients with diabetes and/or hypertension and/or hyperlipidaemia in Malaysia. This knowledge instrument was then used to measure the levels of diabetes-related knowledge of participants in the CORFIS trial. The face and content validity of the DHL knowledge instrument was established by a group of 12 experienced pharmacists and researchers, who went through three drafts of the instrument before producing the final version. The final draft was then piloted on 20 practising community and hospital pharmacists as well as on five diabetic patients in a tertiary hospital, to obtain their feedback concerning the clarity and relevance of the instrument. The DHL knowledge instrument was developed in the English Language as English is understood by a majority of Malaysians. It consists of 28 questions and 5 domains, with each item having true, false and don't know options. The 5 domains are: diabetes (5 items), hypertension (5 items), hyperlipidemia (5 items), medications (8 items) and general issues (5 items).
Patients with type 2 diabetes, currently on medications for their diabetes and who could communicate in English were included in the study. Excluded were children (< 18 years of age) and those who were not taking any medication for diabetes. Patients answered the DHL knowledge instrument twice: at baseline and 4 weeks later.
To obtain a more objective assessment on the validity of the DHL knowledge instrument, pharmacists were recruited (from a tertiary hospital) as the professional group. Pharmacists were expected to have higher knowledge of diabetes than patients. The pharmacists answered the DHL knowledge instrument once only.
Patients were recruited from the diabetes clinic of a tertiary hospital in Kuala Lumpur, Malaysia. Baseline information such as demographic data, medical history, life style and medication history were collected. The questionnaire was administered whilst the patient was waiting to see the doctor. Patients completed the questionnaire themselves, after instructions were given by the researcher. Participants took about 10-15 minutes to complete the DHL knowledge instrument. The completed questionnaire was checked by the researcher to ensure that all questions had been answered. The DHL knowledge instrument was administered again to the same group of patients after one month, during their follow up visit at the clinic. Patients were questioned on any significant changes or events that had occurred within the one month interval and all changes were documented. This study was approved by the Medical Ethics Committee of the hospital under study.
All data were entered and analyzed using the Statistical Package for Social Sciences (SPSS) version 15. The answers for each question in the knowledge questionnaire was scored as true, false or don't know. Analyses was performed by scoring 1 for a correct response and 0 for an incorrect or don't know response. The total score was converted into percentage, ranging from 0 to 100 percent. Each domain score was also analyzed by summing the scores of correct answers and converted into a scale of 0 to 100. Zero indicates the lowest level of knowledge, whilst 100 indicates the highest level.
Analysis of the individual (dichotomous) items
The McNemar's test
This test was used to examine the test-retest reliability on the individual items.
Flesch reading ease
The Flesch reading index is a tool for estimating the reading comprehension level necessary to understand a written document based on the average number of syllables per word and the average number of words per sentence. Scores range from 0 to 100 indicates the level of difficulty in understanding the document, with lower numbers indicating greater difficulty. An average document should have a score of 60-70 .
The difficulty factor is defined by the proportion of patients selecting the right answer to that question and is calculated by the number of correct response divided by the total number of responses. It is a measure of how difficult the question was. The higher the difficulty factor, the easier the question is. A value of higher than 0.75 is deemed to be poor as the question may be too easy. Items with difficulty values between 0.3 and 0.7 are most effective. The optimal level should be 0.5 .
The internal consistency of the DHL knowledge instrument was assessed using Cronbach's α coefficient. Cronbach's α value ≥ 0.70 is considered as having good internal consistency. If omitting an item increases Cronbach's α significantly, then excluding the item will increase the homogeneity of the scale .
Analysis of the different domain scores
The mean knowledge score [standard deviation (SD)] was calculated for each domain. This score reflects the level of knowledge of each domain. Since the data obtained were not normally distributed, non-parametric tests were used.
The Wilcoxon Signed-rank test was used to compare if there was any significant difference between the test and retest results. The Mann-Whitney U test was used to compare if there was any significant difference between the patient and the professional group.
Spearman's rho was used to determine correlation coefficient of the domain scores of the patient group at test-retest.
Factors associated with knowledge scores
Spearman's rho and Chi square tests were used to determine if there were any relationship between demographic factors and knowledge scores for continuous and categorical variables, respectively. A p value of < 0.05 was considered as statistically significant.
A total of 117 participants were recruited: 77 (65.8%) patients and 40 (34.2%) pharmacists (professionals). Of the 77 patients, 37 (48.1%) were female. Sixteen patients (15.5%) were excluded in the test-retest analysis as they did not come for the second follow-up visit even after being reminded through telephone calls, leaving a total of 61 patients. Demographic characteristics of patients are as shown in Table 1.
Psychometric properties of the DHL knowledge instrument
Flesch reading ease was 60, which was considered as satisfactory. Internal consistency for the overall DHL instrument was excellent (Cronbach's α = 0.791).
Corrected item-total correlations were first used to identify items which did not agree well with the other items. Item-total correlations should exceed 0.2 to be considered as acceptable . Using this criterion, items 5, 7, 9, 17, 19 and 28 did not meet this requirement (Table 2). The Cronbach's α value for each item that represented the effect of removing that item from the calculation of the overall Cronbach's α value was also computed. The results showed that the internal consistency of the DHL knowledge instrument remained close to the overall Cronbach's α of 0.791 with the removal of any of these items. Therefore, all 28 items were retained. The difficulty factor for most of the items was satisfactory (0.14-0.92) with a mean(SD) of 0.74(0.21). Out of 28 questions, 8 had a difficulty factor of less than 0.75. This indicates that the knowledge instrument was moderately easy.
Test-retest reliability was assessed in 61 patients after a 4-week interval and all domains showed high correlation coefficients (0.391-0.559, p < 0.001). Out of 28 items, only two items (items no. 15 and 19) were found to be significantly different (Table 2). There was also no difference between the test-retest scores for all domains and the overall score (exceptions were the domains on hypertension and hyperlipidaemia) (Table 3).This indicates that the DHL knowledge instrument has achieved stable reliability.
Baseline diabetes-related knowledge levels of patients and professionals
Table 3 shows the baseline knowledge scores for the patient and the professional group. The mean scores for diabetes, hypertension, hyperlipidaemia and medication were significantly lower in the patient group compared to the professional group, which then affected the overall mean(SD) knowledge score [74.4(14.9) versus 93.8(6.5), p < 0.001] (Table 3). Patients had fairly good knowledge on diabetes and its general issues, followed by knowledge on medications, hypertension and hyperlipidemia. The higher the education level of the patient, the better their knowledge score (Spearman's rho = 0.377, p = 0.001). Other demographic factors such as gender, age, BMI, ethnicity, monthly income, employment status, duration of diabetes and exercise, were not associated with knowledge scores.
Comparison of the DHL knowledge instrument with other validated instruments
The DHL instrument was compared with other validated instruments for assessing knowledge on diabetes (Table 4). The psychometric properties of the DHL knowledge instrument were similar to these other validated instruments. For example, the Flesch reading ease (or equivalent) was around 60 and Cronbach's alpha value was above 0.7.
The DHL developed in this study performed satisfactorily for most of the psychometric components analysed.
The DHL knowledge instrument had a Flesch reading ease of 60. This is within the preferred range score of 60-70, indicating that this instrument is suitable for the average adult to read. Other validated instruments like the DKT  and the PCQ  were found to have a 6th Flesch Kincaid grade level, which indicates that the text is expected to be understandable by an average student in the 6th grade (usually around ages 11-12 in the United States of America). The overall Cronbach's α was good (0.791).
The difficulty factor for most of the items was satisfactory (0.14-0.92) with a mean(SD) of 0.74(0.21). Out of 28 questions, 8 had a difficulty factor of less than 0.75. This indicates that the knowledge instrument was easy for patients to answer, which was further substantiated by the mean baseline knowledge score of 74.4(14.9)%. Other studies on the validation of instruments that assessed diabetes-related knowledge obtained lower scores which ranged from 44.4-61.0% [9, 10, 15]. The only exception was the study by Koontz et al.  which obtained a score of 87.0%. Patients in the present study showed the highest knowledge on diabetes [86.0(19.2)%], and least knowledge on hyperlipidemia [56.6(23.3)%]. This finding is not unusual as not all diabetic patients have hyperlipidemia or hypertension and hence they may not have been provided with information on these conditions. However, these three diseases are inter-related, and patients should know about the cardiovascular risk factors associated with diabetes.
The reliability of the DHL was excellent as shown by the test-retest results. Out of 28 items, only 2 items showed a significant difference in the test-retest results which affected the domain scores on hypertension and hyperlipidaemia. Item 15 ("Omega-3 supplements can reduce "bad" cholesterol more than the medicine given by the doctor") showed a significant improvement in the retest, indicating that patients may have checked the information after the initial test. As for item 19 ("People with type 1 diabetes cannot depend on tablets or oral type of medicines to control their blood sugar"), less patients answered these statements correctly at retest. A possible reason is that the patients "guessed" the answers correctly during the initial test, but could not remember their previous answer and hence "guessed" wrongly the second time.
The overall knowledge scores of the DHL instrument were higher in the professional group (pharmacists) than in the patient group (93.8% versus 74.4%) as well as in all domains. Pharmacists are expected to have higher level of knowledge on diabetes and its related issues compared to patients. This indicates that the DHL instrument is a valid instrument that can be used to differentiate the knowledge levels of diabetes amongst its participants.
One of the limitations of this study was the small sample size. It has been recommended that the number of participants required should be the number of items (that is, 28 items) multiplied by 5-10. This means that at least 140 participants are required. In addition, the DHL knowledge instrument was only evaluated in the English language but Malaysia is a multi-racial country. Therefore, results of this study may not be applicable to patients who cannot read or understand English. Further studies using the Bahasa Malaysia and Mandarin versions of the DHL knowledge instrument are required to enable the results to be more representative of diabetic patients in Malaysia.
The psychometric analysis showed that the DHL knowledge instrument has a satisfactory difficulty factor and shows good promise to be adopted as an instrument for assessing patients' knowledge about diabetes and its medications as well as other cardiovascular metabolic risk factors in Malaysia. The DHL knowledge instrument can be used to identify individuals in need of educational interventions and to record changes over time as treatment progresses for each patient in Malaysia.
Wild S, Roglic G, Green A, Sicree R, King H: Global Prevalence of Diabetes: Estimates for the year 2000 and projections for 2030. Diabetes Care. 2004, 27: 1047-1053. 10.2337/diacare.27.5.1047.
Zanariah H, Chandran LR, Wan Mohamad WB, Wan Nazaimoon WM, Letchuman GR, Jamaiyah H, Fatanah I, Nurain MN, Helen Tee GH, Mohd Rodi I: DWP1-3 Prevalence of diabetes mellitus in Malaysia in 2006 - results of the 3 rd National Health and Morbidity Survey (NHMS III). Diabetes Res Clin Prac. 2008, 79 (1): 21-
Ramachandra A, Ching R, Snehalatha C: Diabetes in Asia. The Lancet. 2010, 375 (9712): 408-418. 10.1016/S0140-6736(09)60937-5.
Colleran KM, Starr B, Burge MR: Putting diabetes to the test: Analyzing glycemic control based on patients' diabetes knowledge. Diabetes Care. 2003, 26 (7): 2220-2221. 10.2337/diacare.26.7.2220.
McPherson ML, Smith SW, Powers A, Zuckerman IH: Association between diabetes patients' knowledge about medications and their blood glucose control. Res Social Adm Pharm. 2008, 4 (1): 37-45. 10.1016/j.sapharm.2007.01.002.
Funnell MM, Brown TL, Childs BP, Haas LB, Hosey GM, Jensen B, Maryniuk M, Peyrot M, Piette JD, Reader D, et al: National standards for diabetes self-management education. Diabetes Care. 2008, 31 (Suppl 1): S97-104.
Beggan MP, Cregan D, Drury MI: Assessment of the outcome of an educational programme of diabetes self-care. Diabetologia. 1982, 23: 246-251. 10.1007/BF00252849.
Dunn S, Beeney LJ, Hoskins PL, Turtle JR: Knowledge and attitude change as predictors of metabolic improvement in diabetes education. Soc Sci Med. 1984, 31: 1135-1141.
Fitzgerald JT, Funnell MM, Hess GE, Barr PA, Anderson RM, Hiss RG, Davis WK: The reliability and validity of a brief diabetes knowledge test. Diabetes Care. 1998, 21 (5): 706-710. 10.2337/diacare.21.5.706.
Garcia AA, Villagomez ET, Brown SA, Kouzekanani K, Hanis CL: The Starr County Diabetes Education Study: development of the Spanish-language diabetes knowledge questionnaire. Diabetes Care. 2001, 24 (1): 16-21. 10.2337/diacare.24.1.16.
Villagomez ET: Health beliefs, knowledge and metabolic control in diabetic Mexican American Adults. 1989, The University of Texas Health Science Center
Dunn SM, Bryson JM, Hoskins PL, Alford JB, Handelsman DJ, Turtle JR: Development of the diabetes knowledge (DKN) scales: forms DKNA, DKNB, and DKNC. Diabetes Care. 1984, 7 (1): 36-41. 10.2337/diacare.7.1.36.
Collins GS, Mughal S, Barnett AH, Fitzgerald J, Lloyd CE: Modification and validation of the Revised Diabetes Knowledge Scale. Diabetic Medicine. 2011, 28 (3): 306-310.
Koontz MB, Cuttler L, Palmert MR, O'Riordan M, Borawski EA, McConnell J, Kern EO: Development and validation of a questionnaire to assess carbohydrate and insulin-dosing knowledge in youth with type 1 diabetes. Diabetes Care. 2010, 33 (3): 457-462. 10.2337/dc09-0390.
Huizinga MM, Elasy TA, Wallston KA, Cavanaugh K, Davis D, Gregory RP, Fuchs LS, Malone R, Cherrington A, Dewalt DA, et al: Development and validation of the Diabetes Numeracy Test (DNT). BMC Health Serv Res. 2008, 8: 96-10.1186/1472-6963-8-96.
Bradley C: Measurement of Diabetes Knowledge- The development of The DKN Scales. Handbook of Psychology and Diabetes. 1994, Amsterdam, Netherlands: Harwood Academic Publishers
Torres HC, Virginia AH, Schall VT: Validation of Diabetes Mellitus Knowledge (DKN-A) and Attitude (ATT-19) Questionnaires. Rev Saude Publica. 2005, 39 (6): 906-911.
Ahmad B, Ramadas A, Quek KF: The Development and Validation of Diabetes Knowledge Questionnaire for the Indigenous Population in Malaysia. Med J Malaysia. 2010, 65 (4): 274-277.
Al-Qazaz HK, Hassali MA, Shafie AA, Sulaiman SA, Sundram S: TRANSLATION AND VALIDATION OF MICHIGAN DIABETES KNOWLEDGE SCALE INTO MALAYSIAN VERSION. Value in Health. 2010, 13 (7): A507-A507.
Readibility formulas: Flesch reading ease. [http://www.readabilityformulas.com/flesch-reading-ease-readability-formula.php]
University Testing Services, Arizona State University: A guide to interpreting the item analysis report. [http://www.asu.edu/uts/pdf/InterpIAS.pdf]
Cronbach LJ: Coefficient alpha and the internal structure of test. Psychometrika. 1951, 16: 297-10.1007/BF02310555.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/12/18/prepub
The authors would like to thank: Associate Prof. Tey Nai Peng (Department of Applied Statistics, Faculty of Economics and Administration University of Malaya, 50603 Kuala Lumpur, MALAYSIA) for his advice on the statistical methods used, The Pharmacist Research Team of the Community-Based Cardiovascular Risk Factors Intervention Strategies (CORFIS) Trial for designing and developing the DHL knowledge instrument, as well as all participants of this study.
The authors declare that they have no competing interests.
PSML researched data, contributed to discussion and wrote the manuscript; SSC moulded the study, designed the questionnaire, contributed to discussion and reviewed/edited the manuscript; CHT collected and researched data, contributed to discussion and reviewed/edited the manuscript; SPC contributed to discussion and reviewed/edited the manuscript. All authors read and approved the final manuscript.
Siew Siang Chua, Ching Hooi Tan and Siew Pheng Chan contributed equally to this work.
About this article
Cite this article
Lai, P.S., Chua, S.S., Tan, C.H. et al. Validation of the diabetes, hypertension and hyperlipidemia (DHL) knowledge instrument in Malaysia. BMC Med Res Methodol 12, 18 (2012). https://doi.org/10.1186/1471-2288-12-18
- Knowledge instrument