- Research article
- Open Access
- Open Peer Review
Objectively-assessed outcome measures: a translation and cross-cultural adaptation procedure applied to the Chedoke McMaster Arm and Hand Activity Inventory (CAHAI)
BMC Medical Research Methodologyvolume 10, Article number: 106 (2010)
Standardised translation and cross-cultural adaptation (TCCA) procedures are vital to describe language translation, cultural adaptation, and to evaluate quality factors of transformed outcome measures. No TCCA procedure for objectively-assessed outcome (OAO) measures exists. Furthermore, no official German version of the Canadian Chedoke Arm and Hand Activity Inventory (CAHAI) is available.
An eight-step for TCCA procedure for OAO was developed (TCCA-OAO) based on the existing TCCA procedure for patient-reported outcomes. The TCCA-OAO procedure was applied to develop a German version of the CAHAI (CAHAI-G). Inter-rater reliability of the CAHAI-G was determined through video rating of CAHAI-G. Validity evaluation of the CAHAI-G was assessed using the Chedoke-McMaster Stroke Assessment (CMSA). All ratings were performed by trained, independent raters. In a cross-sectional study, patients were tested within 31 hours after the initial CAHAI-G scoring, for their motor function level using the subscales for arm and hand of the CMSA. Inpatients and outpatients of the occupational therapy department who experienced a cerebrovascular accident or an intracerebral haemorrhage were included.
Performance of 23 patients (mean age 69.4, SD 12.9; six females; mean time since stroke onset: 1.5 years, SD 2.5 years) have been assessed. A high inter-rater reliability was calculated with ICCs for 4 CAHAI-G versions (13, 9, 8, 7 items) ranging between r = 0.96 and r = 0.99 (p < 0.001). Correlation between the CAHAI-G and CMSA subscales for hand and arm was r = 0.74 (p < 0.001) and r = 0.67 (p < 0.001) respectively. Internal consistency of the CAHAI-G for all four versions ranged between α = 0.974 and α = 0.979.
The TCCA-OAO procedure was validated regarding its feasibility and applicability for objectively-assessed outcome measures. The resulting German CAHAI can be used as a valid and reliable assessment for bilateral upper limb performance in ADL in patients after stroke.
Language is a historically developed form of human communication that reflects culture, traditions, and customs of a country and its inhabitants. It is specific for each country and contains words, symbols and characters or syntax that have a unique meaning for its users . Thus, standardised translation and cross-cultural adaptation (TCCA) procedures are vital to describe language translation, cultural adaptation, and to evaluate quality factors of transformed outcome measures. For example, 'Please dial 144.' is one of the activities that will be assessed with the German version of Chedoke McMaster Arm and Hand Activity Inventory (CAHAI-G). Modifying the telephone number is a cultural adaptation from the original Canadian English CAHAI developed by Barreca and colleagues in 2004 .
In 2009, PubMed has indexed about 22 articles describing a TCCA procedure for patient-reported outcome measures (PRO). Different TCCA-PRO procedures have been developed by research groups [3–6], in particular to translate and adapt assessments in several other languages and implement international multicentre studies. Acquadro and colleagues carried out an extensive literature review and found 17 TCCA-PRO guidelines, and 22 articles that reviewed TCCA-PRO requirements and strategies, and gave recommendations for TCCA-PRO procedures . Nevertheless, those investigations focused on PRO-based assessments, which are subjectively reported by patients. Assessments for objective scoring of assessors were not specifically addressed.
Whereas the development of TCCA-PRO requires an understanding of the general population, objectively-assessed outcome measures (TCCA-OAO) require knowledge of profession-specific vocabulary. The main focus for TCCA-OAO is to explicitly describe the task to be performed by patients as well as the objective scoring instructions for assessors. To our knowledge, no official TCCA guideline exists that specifically addresses OAOs. It is nevertheless essential to apply a standardised procedure for these assessments to 1) carry out translations, 2) adapt test instructions to region-specific and profession-specific vocabulary, 3) review and test the translation for comprehensiveness and practicability by professionals themselves, and 4) evaluate quality factors for its use in a foreign country. These goals have been addressed for TCCA-OAO in this study by adapting a multistep procedure that was recommended by Acquadro et al. .
The aim of this study was to develop and evaluate a procedure for TCCA-OAOs that can provide a generic guidance and standard process for OAOs. The proposed procedure was evaluated by TCCA of the English CAHAI to a general German-speaking audience regarding inter-rater reliability and convergent construct validity of the German version.
Regaining arm and hand function is one of the most frequently named aims for patients after stroke .To this end, OAO measures are essential to set appropriate rehabilitation goals, to record the patient's treatment course, and to evaluate service delivery. OAOs are also important to compare health status among different nations, for international health campaigns, and development of treatment options. Barreca and colleagues developed the CAHAI focussing on upper limb function assessment in patients after stroke. The CAHAI items are based on interviews performed with patients having a hemiparesis as well as their caregivers. In total 751 items have been generated, which finally resulted in 13 bilateral ADL-relevant tasks . Quality factors of CAHAI have been investigated showing a high inter-rater reliability (ICC = 0.98), a minimal detectable change of 6.3 points, and a high correlation with the Action Research Arm Test (ARAT, r = 0.93) and the CMSA (subscale arm and hand r = 0.81) to support convergent validity . Subsequently three shorter versions of CAHAI-13 were developed by the original authors: CAHAI-7, CAHAI-8, and CAHAI-9 . Patients can reach a score for CAHAI 7 ranging between 7 to 49, for CAHAI 8 between 8 to 56, for CAHAI 9 between 9 to 63, and for CAHAI 13 between 13 to 91[8, 9].
Using the proposed TCCA procedure, a German version (CAHAI-G) was derived in this study. Subsequently inter-rater reliability and convergent construct validity of the German version was determined in a patient study. Feasibility and applicability of the TCCA procedure was confirmed using these quality factors.
A formal TCCA-OAO procedure was developed and implemented for CAHAI. The detailed procedure, implementation, and CAHAI-G validation is described as follows.
The translation and cross-cultural adaptation procedure (TCCA)
An eight-step procedure for TCCA-OAO was developed based on the TCCA-PRO procedure proposed by Beaton et al.  and Acquadro et al.'s review recommendations . Both articles suggested following a generic systematic multi-step procedure including more than one forward translation to account for variability in language interpretation. Compared to Beaton's 6-step procedure, the TCCA-OAO was modified and extended to emphasise importance and time needed to perform standardised high-quality translations. Table 1 summarised all individual steps of TCCA-OAO.
The widely known and accepted back translation approach was assumed in Step 4 as no scientific evidence exists to favour a two-panel approach .
The expert committee meeting, suggested by Beaton et al.  for TCCA-PRO, was omitted since informed forward translators were considered and an additional comprehensiveness check was performed by a person of the target profession. Moreover, in contrast to Beaton et al., all translations and necessary material was checked by the original contributors before pretesting with patients. With this approach conceptual errors and misleading scoring of the tested motor functions were avoided.
Furthermore, a TCCA project manager should be made responsible for the whole TCCA procedure, to oversee each step, to make sure that all actors work carefully, to document each step and decision, and to stay in contact with the original authors if discrepancies emerge.
Implementation of the TCCA-OAO procedure
The English CAHAI was forward translated by two native German speakers with excellent English language skills, who were aware of the study objectives. As suggested by Beaton et al. one of the translators was not a member of the target profession (uninformed). None of the 13 tasks have been modified except task 2. To fulfil task 2, patients are asked to dial a region-specific emergency number.
Both translations were synthesised and checked for comprehensiveness by a professional of the target health profession.
The produced test documents were adapted to the clinic's layout style. They were checked for correct grammar and typography by a secretary.
The merged German version was backward translated by a bilingually grown up medical doctor.
The merged German version, the backward translated English version, and photographs of all bought and manufactured objects used in the assessment were sent to the original authors for review.
All CAHAI-G documents were checked for potentially necessary corrections or adaptations by a therapist after receiving permission from the original authors.
The German version was pre-tested by three occupational therapists with four patients before starting a patient study to determine practical feasibility and applicability.
A validation study was performed for CAHAI-G to assess the quality factors.
The CAHAI and CMSA outcome measures and material
The CAHAI contains 13 real-life items scored from one to seven (highest score). Scoring represents the independence of patients to perform stabilisation or manipulation in activities of daily living (ADL) with the affected upper limb. Score one represents total dependence from another person, whereas score seven shows patient's independence without time or safety concerns, necessary splints or devices. After translating CAHAI 13 into CAHAI-G 13 all items required for CAHAI-G 7, 8, and 9 were available. As in the Canadian CAHAI 7, 8, and 9 the German versions (CAHAI-G 7, 8, 9) contain the corresponding items. The officially translated and approved German version (CAHAI-G 13) was used in the patient study. Due to the broad ADL focus of CAHAI, the complete version requires more than 20 household materials, e.g. fork, knife, plate, telephone, glasses, toothbrush. All necessary materials were described in the translated manual regarding their size, weight, shape, and function. The scoring sheet for CAHAI-G (13 items) is presented in the appendix (Additional file 1). All additional scoring sheets (CAHAI-G 7, 8, 9) and the assessment manual can be obtained from the corresponding author.
Additionally, patient's arm and hand motor functions were assessed with the Chedoke-McMaster Stroke Assessment (CMSA) [11, 12]. The CMSA evaluates physical impairment and activity level of stroke patients. The impairment subscales (postural control, arm, hand, leg, foot) were measured on a seven-point scale (1 = low, 7 = high) according to seven stages of motor recovery . Additionally, shoulder pain of the affected body side was evaluated on the same seven-point scale using a further subscale. In this study only subscales for arm, hand, and shoulder pain were analysed. The CMSA was shown to be a valid and reliable assessment [11, 12]. CMSA requires material only for the highest level of motor function in subscale hand: a tennis ball, a cup, and a can with a volume of 250 ml.
CAHAI-G validation study (Step 8)
Participants and study setting
Inpatients and outpatients of the occupational department were recruited for the study if eligible between May and September 2008. The selection criteria are summarised in Table 2. The cantonal ethics committee of the canton Aargau, Switzerland, approved the study as part of a larger project. After giving written informed consent patient's performance of the CAHAI-G 13 were tested and scored by the second author.
Test evaluation and assessor training
All CAHAI assessors were trained by the second author using video material and the translated manual before starting the pre-testing and the patient study. Four patients previously assessed by the second author were scored during a training video rating by the other two assessors. Those patients were not included in the study. Discrepancies, comments and questions regarding instructions or patient scoring were discussed among all assessors. In total, each assessor of the CAHAI underwent a training of ten to 13 hours.
The assessor of CMSA received a two day intensive training course including fundamentals of CMSA, its practical application, and patient testing. All involved occupational therapists had a six to ten years experience in rehabilitation of patients after a lesion of the central nervous system. All test sessions were videotaped.
Test situation and video rating
Patients were tested twice within 31 hours: once with the CAHAI and once with the CMSA. As the CAHAI has to be performed while sitting, except for task 12 and 13, the therapist was sitting next to the patient's affected side to assist if necessary. The task was initially performed and shown by the therapist using hand and arm corresponding to the patient's affected side. After observing the therapist, patients performed each task themselves from a standardised starting position as described in the manual. Two other trained occupational therapists rated videos independently. They were allowed to view the video as often as necessary to score each task.
After the CAHAI evaluation, the patient's arm and hand motor function, and shoulder pain were evaluated by a trained assessor blinded to the CAHA-G scoring. For this purpose, subscales arm, hand, and shoulder pain of the CMSA were used. Patients were given two attempts to perform each task for both assessments.
Descriptive data was calculated representing frequencies, means, and standard deviations for patient's personal data and outcome measures where appropriate. Convergent construct validity was estimated by calculating Spearman's rank correlation coefficient for correlation of the total CAHAI-G score and the CMSA subscales for hand, arm and shoulder pain. Corresponding p-values were derived.
Inter-rater reliability was determined by calculating intraclass correlation coefficients (ICCs) using a two-way random model with absolute agreement and 95% confidence interval (CI) . Values for single measure reliability are provided. Cronbach's alpha was calculated to describe the internal consistency of the CAHAI-G. All statistical analyses were performed with the statistical package for social sciences (SPSS) version 16 for all four versions of CAHAI-G: CAHAI-G-7, CAHAI-G-8, CAHAI-G-9, and CAHAI-G-13. Alpha was set at p ≤ 0.005.
All eight steps were followed as proposed in the TCCA-OAO procedure. We noticed a difference between both forward translations regarding the used vocabulary. The synthesis step was very important to produce one sound forward translation. Therefore, we recommend informing forward translators about the topic or choosing translators from the target-profession.
Layout adaptations and verifying grammar and typography required more time than expected. Both performed checks were absolutely necessary to clarify understanding and interpretation of tasks and scoring instructions.
Study population and assessment scoring
During the recruitment phase 23 patients (mean age 69.4, SD 12.9; 6 females) met the selection criteria and were included in the study. Time since the event ranged between 26 days to eight years (mean 1.5 years, SD 2.5 years). In nine of 22 right-handed patients the left body side was affected. Fifteen patients were reliant on a wheelchair, three patients used a stick, one patient used a rollator walker, and four patients did not use any walking devices. Patients mean scores of all three raters for the CAHAI-G version with seven, eight, nine and 13 items, and the corresponding scores for CMSA subscales hand, arm, and shoulder pain are shown in Table 3. For all single CAHAI-G tasks (one to 13) mean scores for all three raters are provided in Table 4.
All CAHAI-G raters used the whole range of the scoring scale ranging from one to seven to evaluate bilateral arm function in all 23 patients. The most agreements were reached for task 13 ('carry the bag up the stairs'), and the least for task 10 ('zip up the zipper'). Duration of assessment implementation/performance ranged between ten minutes for the CAHAI-G 7 to 35 minutes for the CAHAI-G 13. In 18 out of 23 patients the scoring varied between one to three points for all 13 tasks and among all three raters. In five out of 18 patients both video ratings reached the same scoring but deviated from the direct rating during patient performance in all tasks. The highest congruence among all raters was found in patients achieving a scoring between 13 to 40 points and between 70 to 91 points. The range between 40 to 70 points showed the most variability
Evaluated motor functions with CMSA ranged from two to seven in all patients for subscale hand and shoulder pain. Scores for subscale arm motor function have been evaluated at level two, three, four, five, and seven.
Internal consistency, ICCs, and correlations for all four CAHAI-G versions are shown in Table 5.
Inter-item correlations range between r = 0.414 (for 'Clean a pair of eye glasses' and 'Place container on table') and r = 0.945 (for 'Open a jar of coffee' and 'Call 144').
Convergent construct validity
For convergent construct validity a close relation of CAHAI-G and CMSA subscales arm and hand was hypothesised. The correlation coefficients show a large correlation for CAHAI-G and CMSA subscale hand (r = 0.74) and a moderate correlation for CAHAI-G and CMSA subscale arm (r = 0.67).
By applying the proposed TCCA-OAO procedure for CAHAI a German version of the task instruction manual and scoring descriptions was developed. The eight-stage procedure allowed for a resource-efficient method regarding time and personnel (5 people) involved compared to Beaton et al. (9 people) and Swain-Verdier et al. (6 to 8 people) [3, 7, 10]. Different from PRO, OAOs benefit from an explicit instruction and scoring description for therapists, which is profession-specific and does not require to be understood by the general population. The control checks of the documents between forward and backward translation by experienced clinicians of the target profession (steps 3 and 6) helped to achieve this goal. With this approach, the particular risk for interpretation differences in OAOs was reduced. A standardised TCCA-OAO is essential for error-free assessment adaptations. With the proposed TCCA procedure five of the six levels of equivalence of Stewart and Napoles-Springer (cited in Acquadro ) have been reached: 1) conceptual, 2) semantic, 3) operational, 4) measurement, and 5) item equivalence. The sixth level (criterion equivalence) was not achieved due to the fact that the quality factors were evaluated in a homogeneous patient group.
It is widely accepted that more than one forward translation should be done to create translated versions from different perspectives [1, 3, 6, 7]. However, imposed effort and cost for a TCCA procedure standard must be carefully reviewed. A balance between TCCA quality and invested resources can ensure that standards are considered in clinical practice globally. We are not in line with the suggestion that required back translators should be uninformed native speakers of the original language, living in the target country . Finding adequate translators can require large efforts and raise costs. Moreover, given that the backward translation is validated by the original authors, this requirement appears unnecessary. Thus, we suggest that back translators can be informed native speakers of the target language with excellent language skills of the source language or applying a two-panel approach suggested by Swaine-Verdier et al. . Furthermore, we belief, that the whole TCCA procedure for OAOs is accelerated by having informed forward and backward translators, which was a design choice in this work.
Results of the patient study revealed a very high inter-rater reliability. This was achieved by establishing strict video recording rules during the practical performance of the CAHAI-G: (1) sitting position of the patient and the therapist who was sitting at the affected side of the body and 2) having a second therapist recording the performance from wide-angle and close-up distances in frontal view to emphasise small patient movements. Moreover, the assessor could watch videos as often as necessary.
Our results showed very similar values for inter-rater reliability, validity and internal consistency compared to the original CAHAI which support the suggested TCCA procedure. Larger ICC values obtained for CAHAI-G could be attributed to the video rating performed versus direct performance scoring done in the original. Convergent construct validity could be demonstrated at high to moderate correlations with CMSA subscales for arm and hand.
The small sample size of 23 patients in a subacute or chronic stage of their disease might be a limitation of the current study. Furthermore, we did not sum up CMSA subscale hand and arm scores for the validity calculation, which could have lowered correlation results for CAHAI-G compared to the validation study of the original version. We have found a wide range of inter-item correlations, which we account to the smaller sample size as well.
This study was focussed on the translation, validation and inter-rater reliability evaluation of the CAHAI. Further quality factors, e. g. intra-rater reliability and responsiveness, of the German CAHAI version were not addressed in this work. These evaluations of the CAHAI-G will be the focus of future projects. With the help of the current study the CAHAI-G has been successfully introduced to the occupational department in our rehabilitation centre and will be used in clinical routine.
Further studies are needed to confirm the suggested TCCA procedure in other OAO measures.
Various approaches and options could be used to translate and culturally adapt OAOs. Our suggested eight step TCCA-OAO procedure is the first of its kind and is based on reviews of existing PRO guidelines and was applied in a practical example that confirmed its applicability. All four versions of the CAHAI-G (CAHAI-G 7, CAHAI-G 8, CAHAI-G 9, and CAHAI-G 13) can be recommended as reliable and valid performance-based measures to assess bilateral upper limb ADL task performance in clinical practice. They can be utilised with patients after stroke in the sub-acute and chronic phase. The shortest version (CAHAI-G 7) needs 10 minutes to perform and the test material is low priced and easy to provide.
We would like to encourage developers of outcome measurements to play an active role in TCCA or even provide official translations of their outcome measure for researchers in foreign countries.
- ARAT :
Action research arm test
- CI :
- CAHAI :
Chedoke Arm and Hand Activity Inventory
- CAHAI-G :
Chedoke Arm and Hand Activity Inventory - German version
- CAHAI-G 7 :
Chedoke Arm and Hand Activity Inventory - 7 items to test
- CAHAI-G 8 :
Chedoke Arm and Hand Activity Inventory - 8 items to test
- CAHAI-G 9 :
Chedoke Arm and Hand Activity Inventory - 9 items to test
- CAHAI-G 13 :
Chedoke Arm and Hand Activity Inventory - 13 items to test
- CMSA :
Chedoke-McMaster Stroke Assessment
- ICC :
Intraclass correlation coefficient
- OAO :
- PRO :
Patient-reported outcome measures
- SPSS :
Statistical package for social sciences
- TCCA :
Translation and cross-cultural adaption
Hoegh MC, Hoegh S-M: Trans-adapting outcome measures in rehabilitation: Cross-cultural issues. Neuropsychological Rehabilitation: An International Journal. 2009, 19 (6): 955-970. 10.1080/09602010902995986.
Barreca S, Gowland CK, Stratford P, Huijbregts M, Griffiths J, Torresin W, Dunkley M, Miller P, Masters L: Development of the Chedoke Arm and Hand Activity Inventory: theoretical constructs, item generation, and selection. Top Stroke Rehabil. 2004, 11 (4): 31-42. 10.1310/JU8P-UVK6-68VW-CF3W.
Beaton DE, Bombardier C, Guillemin F, Ferraz MB: Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000, 25 (24): 3186-3191. 10.1097/00007632-200012150-00014.
McKenna SP, Doward LC: The Translation and Cultural Adaptation of Patient-Reported Outcome Measures. Value in Health. 2005, 89-91. 10.1111/j.1524-4733.2005.08203.x. 8
Guillemin F, Bombardier C, Beaton D: Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993, 46 (12): 1417-1432. 10.1016/0895-4356(93)90142-N.
Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, Erikson P: Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005, 8 (2): 94-104. 10.1111/j.1524-4733.2005.04054.x.
Acquadro C, Conway K, Hareendran A, Aaronson N: Literature review of methods to translate health-related quality of life questionnaires for use in multinational clinical trials. Value Health. 2008, 11 (3): 509-521. 10.1111/j.1524-4733.2007.00292.x.
Barreca SR, Stratford PW, Lambert CL, Masters LM, Streiner DL: Test-retest reliability, validity, and sensitivity of the Chedoke arm and hand activity inventory: a new measure of upper-limb function for survivors of stroke. Arch Phys Med Rehabil. 2005, 86 (8): 1616-1622. 10.1016/j.apmr.2005.03.017.
Barreca S, Stratford P, Masters L, Lambert CL, Griffiths J, McBay C: Validation of Three Shortened Versions of the Chedoke Arm and Hand Activity Inventory. Physiother Can. 2006, 58: 148-156. 10.3138/ptc.58.2.148.
Swaine-Verdier A, Doward LC, Hagell P, Thorsen H, McKenna SP: Adapting quality of life instruments. Value Health. 2004, 7 (Suppl 1): S27-30. 10.1111/j.1524-4733.2004.7s107.x.
Gowland C, Stratford P, Ward M, Moreland J, Torresin W, Van Hullenaar S, Sanford J, Barreca S, Vanspall B, Plews N: Measuring physical impairment and disability with the Chedoke-McMaster Stroke Assessment. Stroke. 1993, 24 (1): 58-63.
Gowland C, VanHullenaar S, Torresin W, et al: Chedoke-McMaster Stroke Assessment: development, validation, and administration manual. 1995, Hamilton (ON): School of Rehabilitation Science, McMaster University
Gowland CA: Staging motor impairment after stroke. Stroke. 1990, 21 (9 Suppl): II19-21.
Shrout PE, Fleiss JL: Intraclass correlations: uses in assessing rater reliability. Psychological bulletin. 1979, 86 (2): 420-428. 10.1037/0033-2909.86.2.420.
Hahn S, Schuster C, Kujawski A, Viehweg S, Zarkowski M, Ettlin T: Translation, validity and inter-rater reliability analysis for the German version of the Chedoke Arm and Hand Activity Inventory (CAHAI-G). European Journal of Neurology. 2009, 16 (s3): 55-334.
Maneesriwongul W, Dixon JK: Instrument translation process: a methods review. J Adv Nurs. 2004, 48 (2): 175-186. 10.1111/j.1365-2648.2004.03185.x.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/10/106/prepub
The authors are grateful to Mrs Baerbel Ebert and Dr. Oliver Amft for forward translation and Dr. Udo Hartl for backward translation of the complete manual. We thank all patients for their time and energy they have spent to support the project. Furthermore, we thank all therapists at the occupational department who performed the data collection: Alexandra Kujawski, Silvia Viehweg, and Michaela Zakowski. We appreciated the efficient cooperation with the original authors Susan Barreca, Lisa Masters and their team during translation, data collection and analysis process. We are thankful for their constant support to answer questions, providing pictures of the necessary test objects, and video material for assessor training.
The authors declare that they have no competing interests.
The publication is based on the Master thesis (MSc.) of the second author.
The work has been presented as a poster at the EFNS conference in Florence 2009 .
CSch has made substantial contribution to conception and design, data analysis and interpretation, and has written the article. SHa was involved in the design process. She did the data collection, data analyses and wrote parts of the manuscript. ThE contributed to the conception and design of the study. He was involved in the data interpretation process, drafted and critically revised the manuscript. All authors have given final approval of the manuscript.