Psychometric properties of two physical activity questionnaires, the AQuAA and the PASE, in cancer patients

Study Sample

Patients were recruited from the departments of Hematology, Oncology, Radiotherapy and Gynaecology of Academic Medical Centre (AMC) and the VU University Medical Center (VUmc) in Amsterdam from January to April 2010. The eligibility criteria were: (a) histologically confirmed primary cancer, treated with (neo adjuvant) chemotherapy, or histologically confirmed (relapsed) hematologic malignancy with no indication of progressive disease, treated with high-dose chemotherapy followed by autologous stem cell transplantation; (b) age between 18 and 70 years; (c) having received the last (active) treatment within 1 year prior to participation into this study, and (d) World Health Organization (WHO) performance status of 0 (asymptomatic; fully active and ambulatory) or 1 (symptomatic but completely ambulatory; restricted in physically strenuous activities but able to carry out light and sedentary activities). Patients who received chemotherapy only as palliative treatment were excluded from participation. The study protocol was approved by the Medical Ethics Committee of both the AMC and VUmc. Patients signed an informed consent statement before participating in the study.

Procedures

Participants were fitted with an accelerometer for 7 consecutive days, see Figure 1. On the 8^th day they completed both the AQuAA and the PASE at their homes (T0). Five days later the questionnaires were completed for a second time (T1). No specific instructions were given with regard to the order in which the questionnaires were to be completed. At T0 or T1, we conducted a Three-Step Test-Interview (TSTI) in a subpopulation. The validity and the reliability study involved the same participants. We aimed to enrol 50 participants into the study since this is considered an adequate sample size for studies assessing the validity and reliability of measurement instruments [12].

Activity Questionnaire for Adults and Adolescents (AQuAA)

The AQuAA is a short, self-report 7-day PA recall questionnaire [17], designed to assess daily PA and sedentary behaviour in adults and adolescents. It is divided into five categories; commuting activities, PA at work/school, household chores, leisure time activities and active sports. Each activity can be complemented with its frequency (number of days in the last week), duration (hours, minutes) and the perceived intensity (low, medium or high). Five main outcomes can be calculated: a total PA score (the AQuAA score in Metabolic Equivalent of Tasks (MET)*min/week, including all activities ≥ 2 METs), and the total time (in min/week) spent on sedentary (< 2 MET), light (2-4 METs), moderate (4-6.5 METs) and vigorous (> 6.5 METs) intensity activities. In healthy adults, test-retest reliability was fair to moderate (Intraclass correlation coefficients (ICCs) = 0.49 to 0.60) on all outcomes except for time for time spent on vigorous activities (ICC=-0.005) [17], and the correlations with the ActiGraph accelerometer were low and not significant (r_s=-0.16 to 0.15) [17].

Physical Activity Scale for the Elderly (PASE)

The PASE is a brief, self administered 7-day recall questionnaire specifically designed to assess PA in older adults [18]. The PASE consists of questions on leisure time, household and work-related activities. The frequency of these activities are recorded as never, seldom (1-2 days/week), sometimes (3-4 days/week), or often (5-7 days/week). The duration of activities is categorized as less than 1 hour, between 1 and 2 hours, between 2 and 4 hours, or more than 4 hours. Paid or volunteer work, except for work that involves mostly sitting activities such as office work, is categorized as less than 1 hour, between 1 and 4 hours, between 5 and 8 hours, or more than 8 hours [21]. The total PASE sum score is computed by multiplying the amount of time spent on each activity (in hours/week) by the empirically derived item weights and summing over all activities. In healthy elderly the PASE has been shown to have high test-retest reliability (r_p = 0.84) [18] and reasonable validity as compared with the doubly labelled water method (r_s = 0.68) [22].

ActiGraph Accelerometer

PA was objectively assessed using the ActiGraph accelerometer (type ActiTrainer, Manufacturing Technology Inc., Pensacola, FL), a small (8.5 × 3.5 × 1.5 cm) and lightweight (51 g) PA monitor. The accelerometers were initialized according to the manufacturer's specifications, and PA was recorded in epoch intervals of 15 seconds. The participants were instructed to wear the accelerometer for seven consecutive days on their right hip during all waking hours. Since the accelerometer is not waterproof, it was not worn during water-based activities. A simple journal was provided to register the time of waking up and going to sleep, and any other instances during the day at which the accelerometer was not worn. We converted the uni-axial vertical accelerations measured by the ActiGraph into activity counts per minute. The Freedson regression-based equation was applied to categorize the activities into sedentary (< 100 counts/min), and light (< 3.0 METs or 100-1951 counts/min), moderate (3.0-5.9 METs or 1952-5724 counts/min) and vigorous (≥ 6.0 METs or ≥ 5725 counts/min) intensity [23]. A wearing day was considered valid if data was collected for at least 600 minutes (10h) that day. Non-wearing time was defined as 60 minutes of consecutive zero counts [24]. Data collected for at least 5 of the 7 wearing days were included in the validity analysis. All accelerometer data were analysed using the MeterPlus Version 4.2 software from Santech, Inc. http://www.santechhealth.com. Accelerometry has been shown to be a reasonably valid method to objectively assess PA in adults [17].

Test-retest Reliability

Reliability concerns the degree to which a measurement is free from measurement error [25]. The test-retest reliability of the AQuAA and the PASE was assessed by the extent to which repeated administrations of the instruments in the same subjects and under the same circumstances provided similar results. Since both PA questionnaires had a recall period of 7 days, a time interval of 5 days between repeated measurements was considered appropriate to be short enough to avoid (clinical) changes in PA levels and long enough to prevent recall bias [26].

ICCs were calculated by dividing the variance between patients by the total variance [27]. An ICC value less than 0.40 was rated as poor, 0.40-0.59 as fair, 0.60-0.74 as good and values exceeding 0.75 as excellent [28]. The standard error of measurement (SEM) was calculated by taking the square root of the error variance. The corresponding smallest detectable difference at a 95% confidence level (SDD₉₅) was calculated using the following formula: 1.96 × √2 × SEM [26].

In order to compare the SDD of the AQuAA and the PASE, SDDs were expressed as percentage of the measurement range [29]. To exclude potential outliers, we determined the range (range₉₅) by the differences between the lowest (2.5^th percentile) and highest (97.5^th percentile) observed values for the different measures. The measurement error of an instrument may be considered small enough when the instrument is able to distinguish 7 steps (with a range from 5 to 9) on the measurement range [30]. Therefore, we considered a questionnaire with a SDD₉₅/range₉₅ ratio ≤ 0.20 to be useful for clinical practice [29].

Construct Validity

Construct validity concerns the degree to which an instrument truly measures the construct it claims to measure [12]. Since there is no gold standard for measuring PA, we assessed the construct validity by comparing the AQuAA and the PASE scores with data from the ActiGraph accelerometer [12].

The data were checked for normality using normal probability plots and the Kolmogorov-Smirnov test. Since data were not normally distributed, we calculated Spearman correlation coefficients (r_s) between total activity counts of the ActiGraph accelerometer (counts/min) and total scores of the AQuAA (MET*min/wk) and the PASE (PASE score).

ICCs between the ActiGraph accelerometer, and respectively the AQuAA and the PASE, were calculated for time spent on total physical and sedentary activities (expressed in min/wk). For the AQuAA, additional ICCs were calculated for time (in min/wk) spent on light and moderate-to-vigorous intensity activities. Based on systematic reviews on psychometric properties of PA questionnaires [31, 32], we considered an r ≥ 0.50 as adequate.

Content Validity

Content validity addresses the degree to which an instrument's content adequately reflects the construct to be measured [12]. We used the Three-Step Test Interview (TSTI), an observation-based procedure to identify response problems in self-administered questionnaires [33], to analyse how participants interpreted and responded to the questions. The interview consisted of three consecutive steps; (1) concurrent think aloud, aimed at collecting observational data, (2) focused interview, aimed at remedying gaps in observational data and (3) semi-structured interview, aimed at eliciting experiences and opinions with regard to the questionnaire. The TSTI is a validated pre-testing tool [34] and has previously been used to analyse cognitive processes in cancer patients [35]. Interviews were conducted until "theoretical saturation" was reached, i.e. no new information emerged [34].

The interviews were conducted either at the participants' homes, work or hospital, in accordance with the participants' preferences. Before the interview began, the steps and procedures of the TSTI were explained to the participants. The participants answered both questionnaires, and the order was alternated between subsequent interviews. To further decrease potential bias from answering the first questionnaire, the participants were instructed not to refer to the answers they provided with the first questionnaire while answering the second one.

We constructed three themes with associated questions to identify the participants' opinions of the questionnaires; "Interpretability" ("Is the phrasing of the questionnaires clear to you?"), "Comprehension" ("Are all (of your) daily physical activities addressed in the questionnaires?") and "Preference" ("Which questionnaire were you more comfortable with answering?"). All interviews were audio-taped and transcribed verbatim. A summary of each individual participant's statements regarding the different themes was made and typical verbatim quotations of commonly shared statements were reported.

Test-retest Reliability

Test-retest reliability of the AQuAA score was good; ICC = 0.70. The SDD₉₅/range₉₅ ratios ranged from 0.30 to 0.44, indicating that the AQuAA can distinguish 2 to 3 steps on the observed measurement range.

The ICCs for the PASE were good to excellent (ICC = 0.67 to 0.90. The SDD₉₅/range₉₅ ratios for the PASE were 0.22, 0.15 and 0.79, indicating that the PASE can distinguish 5 steps on the observed measurement range for the PASE sum score, 7 steps for the time spent on total PA and 1 step for the time spent on sedentary activities.

Construct Validity

The Spearman correlation coefficient between the AQuAA score and the ActiGraph was low and not significant (r_s = 0.05, p = 0.716). ICCs were poor for various PA scores (ICC = -0.001 to 0.32), and fair for sedentary activities; ICC = 0.44 (Table 3).

The correlations between the ActiGraph and the PASE were low and not significant (r_s = 0.16, p = 0.279 and ICCs = 0.12 and 0.39) (Table 3).

The median time spent on physical activities was 1348 min/wk for the ActiGraph, 1330 min/wk for the AQuAA and 1609 min/wk for the PASE (see Table 3). 38% of the participants did not meet the American College of Sports Medicine (ACSM) recommendation of at least 150 minutes of moderate-intensity PA per week [36].

Content Validity

Thirteen participants preferred PASE, and three preferred the AQuAA. In general, the pre-structured answers of the PASE facilitated recall as opposed to the open-structure of the AQuAA.

Test-retest Reliability

The test-retest reliability of the AQuAA and the PASE were good to excellent. The reliability of the AQuAA in the current study was higher compared to a previous study among healthy adults [17]. This may partly be caused by the shorter time interval between repeated administrations of the questionnaire in the current study (5 days versus 2 weeks). Alternatively, cancer patients may be more aware of their PA behaviour compared to healthy adults, resulting in better recall. The excellent test-retest reliability of the PASE sum score is in accordance with previously reported studies in healthy elderly [18, 37].

We presented SEM and SDD₉₅ values to indicate the magnitude of measurement error which should be taken into account when judging whether PA levels have really improved over time. Any increase in PA scores exceeding the SDD₉₅ can be attributed, with reasonable confidence, to real improvements in PA level. High measurement errors were observed for both the AQuAA and the PASE. However, based on the calculated SDD₉₅/range₉₅ ratios we judge that the measurement error of both the PASE sum score and the time spent on total PA were sufficiently small (0.22 and 0.15 respectively) to make this questionnaire valuable for clinical practice. For these measurements the PASE is able to distinguish 5 and 7 steps on the observed measurement range. To detect change over time, distinguishing 7 steps (with a range of 5 to 9 steps) is considered adequate [30]. Final judgement about sensitivity to change requires a longitudinal follow-up study.

The good-to excellent reliability of sedentary activities indicate that the AQuAA and PASE might also be useful to assess sedentary behaviours.

Construct Validity

The poor agreement between the ActiGraph accelerometer and the questionnaires do not confirm their construct validity. Previous studies of PA questionnaires in adults and the elderly also showed low correlations with accelerometers [31, 32].

Our results showed low and nonsignificant correlations between the AQuAA and the ActiGraph for the total scores and the time spent on physical activities. These findings are in line with previously published reports of the AQuAA in healthy adults [17], and suggests, similarly to healthy adults [17], that cancer patients may also have difficulty with accurate recall of the duration and intensity of PA during the past 7 days.

In contrast to our findings, Dinger et al. [37] found a significant correlation between the ActiGraph and PASE (r_s= 0.43; p < 0.01). The interview-based administration may have reduced over- or underreporting and misclassification of PA compared to the reliance on self-report in the current study. Otherwise, low correlations may result from the detection of light intensity PA by the accelerometer, while participants may not have realised to be physically active, and consequently did not report it. Furthermore, self-report questionnaires and accelerometers do not measure the exact same construct of PA. Accelerometers provide objective information on PA duration and intensity, whereas the AQuAA and the PASE also provide insight into the types of activities. Relatively high agreement between the accelerometer and sedentary scores assessed by questionnaire indicate their usefulness to assess sedentary behaviours.

Content Validity

The TSTI method showed that participants perceived several problems when answering the AQuAA and the PASE regarding the type, intensity and duration of activities. This is an inherent problem of self-report questionnaires, and may also be associated with this particular study population due to their reduced fitness level and/or increased level of fatigue. Since the participants may perceive certain types of PA as more intensive or maybe compare their current PA level to their pre-diagnosis level, recall bias may have been introduced. This actually became clear during the TSTI, when several participants indicated certain activities to be more intensive than indicated in the questionnaire.

Strengths and Limitations of the Study

This is the first study to combine both quantitative and qualitative methods to assess PA questionnaires in cancer patients. We extensively assessed the test-retest reliability, including measurement errors, and determined the construct and content validity of the questionnaires.

However, this study has several limitations. First, since there is no gold standard to measure PA, the accelerometer was used as comparison measure for the assessment of the construct validity of the questionnaires. However, both instruments have well-known limitations. Waist-worn accelerometers underestimate (light) upper-body movements, such as sweeping and weight-bearing activities, and other daily life activities such as swimming, bicycling and static activities [38]. Nevertheless, the accelerometer is technically speaking a reliable, precise and objective instrument [39]. Self-report measures of PA are limited by factors including social desirability, recall bias, and variations in cognitive and memory processes depending on several factors including age, education and gender [40]. These limitations may have contributed to the discrepancies observed between the ActiGraph accelerometer and the self-report questionnaires in this study. Second, standardized regression equations to calculate time spent on activities in different intensity levels by accelerometry are lacking. Accelerometer cut-off points for PA intensity and sedentary activities are still a matter of debate [41]. Although more recent cut-off points have been published [42], we chose to use the generally known and widely used Freedson cut-off points [23]. This allows for comparison of PA levels with other studies in cancer patients [13, 14, 24]. Researchers should take into account which regression equation and cut-off points were used when comparing studies, since the measured level of PA depends on the choice of cut-off points. Third, due to the broad duration categories, the PASE may not to be able to detect small changes in PA levels. Future studies are therefore needed to assess the responsiveness of the PASE.