Responsiveness of five condition-specific and generic outcome assessment instruments for chronic pain

Patients

The subjects included in the study were all participants of the "Zurzach Interdisciplinary Pain Program" (ZISP) who were suffering either from chronic non-specific back pain (i.e. lumbar, thoracic, cervical, or panvertebral pain syndrome), or fibromyalgia according to the definition of the American College of Rheumatology (ACR), or chronic widespread pain, i.e. generalized musculoskeletal pain syndrome which did not meet the definition criteria of fibromyalgia [21]. The ZISP program is a comprehensive, standardized, four week inpatient pain program at the rehabilitation clinic "RehaClinic", Bad Zurzach, Switzerland and consists mainly of medical care including adapted drug therapy, graded activity exercise, and cognitive behavioral therapy. A detailed description of the program with the inclusion and exclusion criteria has already been published as part of our outcome paper [5].

Measures

The Short Form 36 (SF-36) is the most widely accepted and frequently used generic instrument that comprehensively measures physical, mental and psychosocial health by means of 36 items (questions) that determine 8 scales [22, 23]. The West Haven-Yale Multidimensional Pain Inventory (WHYMPI, abbreviated to MPI) assesses pain and pain-specific consequences in terms of symptoms, activity, behavior, mood, and social relationships on the basis of 51 items that construct 12 scales [24, 25]. The Hospital Anxiety and Depression Scale (HADS) measures anxiety and depression based on 7 items each and is well established in psychology and psychiatry with a long history of application [26, 27]. The Coping Strategies Questionnaire (CSQ) is the tool most often used to assess cognitive and behavioural strategies to tolerate, manage and compensate for pain and their consequences, and is based on 48 items resulting in 8 scales plus 2 additional control items [28–30]. All four instruments are standardized, well tested and widely used – a quick search in MedLine showed 2000–6300 citations for each of the four tools (February 26, 2008). In addition, current pain was assessed by the 11-point Numeric Rating Scale (NRS) ranging from 0 = no pain to 10 = most pain imaginable [31].

On scale level, two instruments can be compared if the items that make up the scales ask about the same domain, i.e. have the same construct [19, 20]. Thus, MPI pain severity was compared with the SF-36 bodily pain and the pain NRS for the assessment of pain. SF-36 role physical together with SF-36 social functioning were compared to MPI interference with pain in the assessment of somatic and psychosocial consequences of the pain disorder covering activities of daily living, work, leisure, and social participation. Function, including ambulation and specific activities (home and outdoor), was covered by SF-36 physical functioning and MPI general activities score; the latter was determined by all 18 activity items as previously described [32]. Affective health/mood (explicitly: happiness, tension, irritability, nervousness, calmness/quietism) was assessed by HADS depression and anxiety scales and compared to SF-36 mental health and MPI negative mood. Control over pain was measured by MPI control pain and CSQ control pain (each by one item). These domains have been previously described and the overlap of their constructs has been tested empirically [19].

Analysis

Assessments were performed at entry into the clinic (baseline) and in the last two days before discharge, i.e. four weeks after entry (follow-up). The scores were determined following the "missing rules" of the instruments, i.e. to determine a score, at least 50% of the items had to be filled out for the SF-36 and 6/7 (86%) for the HADS [22, 27]. For the MPI and the CSQ, where the developers of the questionnaires do not describe missing rules, we used the previously described 2/3 (67%) criteria [5, 6, 30]. The score range was transformed into 0 = maximal pain/no function/worst coping/worst health to 100 = no pain/full function/best coping/best health for all instruments' scores as originally described for the SF-36 to ease comparison between them with the exception of the pain NRS (0 = no pain, 10 = maximal pain) [5, 6, 22]. All analyses were performed using the statistical software package SPSS 16.0 for Windows^® (SPSS Inc., Chicago, IL, USA).

The score difference between follow-up and baseline divided by the standard deviation of the group's baseline scores is defined as effect size (ES), originally introduced as "Glass's delta" [10, 12]. The score difference (follow-up – baseline) divided by the standard deviation of the group's score differences determines the standardized response mean (SRM), originally published as the "Hedge's g" for one sample which is equal to the "Cohen's delta" in this case [12, 14]. The ES and the SRM are the most common measures for responsiveness. Positive values reflect (standardized) improvements in the number of standard deviations of the baseline scores (ES) or the score differences (SRM) (i.e. unit-free) [7, 11]. An ES ≥ 0.80 is considered as large, 0.50–0.79 as moderate, 0.20–0.49 as small, and 0.00–0.19 as very small [10].

To test whether the difference of two responsiveness measures within a certain domain was statistically significant, the "modified Jacknife test" was applied [7, 18]. This method is a linear regression between the difference of the ES or SRM of two comparable scores (e.g. between SF-36 bodily pain and MPI pain severity) as dependent variable and the "centered" ES/SRM of one of the two scales (which scale is not relevant) as independent variable. If the regression's intercept (value of the SRM/ES difference where the centered ES/SRM is equal to zero) is greater or smaller to zero with significance p < 0.050 there is significant difference of the responsiveness of the two scales. For that, the difference of the two ES/SRM are computed in SPSS individually for each patient as well as the "centered" ES/SRM which is equal to the individual ES/SRM minus the (mean) ES/SRM of the whole sample [18].

In multiple pairwise testing of (at least partly) non independent scores (e.g. within the patient-rating of pain), the significance level must be reduced by the number of tested scores (k), i.e. p = 0.05/(k!/(k-2)!*2!) which is well know as the Bonferroni-correction [33]. Thus, the significance level for type I error was set at p = 0.050/3 = 0.017 for comparison of k = 3 instruments (MPI, SF-36, pain NRS in pain and SF-36, MPI, HADS in affective health) and at p = 0.050 for comparison of two instruments.

To quantify the extent of the overlapping constructs within a domain, bivariate Spearman rank correlation coefficients of the baseline scores and the effects (raw score differences baseline → follow-up) were determined for each pair of scales being compared [20].

An additional way to assess the size of effects is to compare the ES with the minimal important difference (MID) for which the estimate is based on the standard error of measurement (SEM) [34]. The SEM in score units is equal to the baseline standard deviation of the scores multiplied by the square root of (1-r), where r is the reliability measure of the scale, usually the intraclass correlation coefficient [34]. The SEM in responsiveness units is therefore equal to the square root of (1-r) for the ES and equal to the baseline standard deviation divided by the standard deviation of the score differences times the square root of (1-r) for the SRM. Note that the SEM in ES or SRM units is independent of the frequency distribution of the scores, i.e. from the sample itself – it is only dependent on the reliability coefficient. We chose the "one-SEM" criterion which means that 1*SEM is an estimate of the MID, the effect which patients (on average) perceive as subjective change [34]. The MID can be used as an estimate for the minimal clinically important difference (MCID) which principally is an anchor-based (on an external criterion) method to assess the smallest effect that patients perceive to be beneficial. As empirically shown, the MID and the MCID often are in the same size [34] or the MCID is even smaller than the MID (see example in the Discussion: [35]). Further information about the importance of the MCID can be found elsewhere [8, 16]. However, the present study compared within-subject and not between-subject effects.

Patients

The cohort consisted of 273 chronic pain patients assessed between 1999 and 2006 whose characteristics have been reported in detail elsewhere [5, 36]. The median pain duration was 60 months (5 years). The mean age was 46.3 years (standard deviation = 10.5, normally distributed) and 79.9% were female. Fibromyalgia was present in 43.2%, chronic back pain in 42.5%, and chronic widespread pain in 14.3% of the cases. There were very few omissions, i.e. complete data pairs (i.e. 273 score pairs baseline – follow-up) were available for most of the scores: the MPI negative mood and the pain NRS had 272, the HADS anxiety 271, the MPI activity and the MPI control each 270, and the CSQ control 267 complete data pairs.

Responsiveness analysis

Comparing the ES and SRM data within the same scale, the differences were mostly small (i.e. |ES-SRM| ≤ 0.10) except for SF-36 role physical, SF-36 bodily pain, the SF-36 mental health, SF-36 mental component summary (MCS), MPI pain severity, HADS depression, and CSQ praying or hoping. In SF-36 role physical, the reason for this may be the high floor effect: The scores at baseline were close to zero which resulted in a small baseline standard deviation and, by that, in a high ES when compared to the lower SRM which is determined by the relatively higher standard deviation of the difference. The effects of the sample were slightly or much higher than the estimated MCID (3^rd and 2^nd columns from the right of Table 1) of the Pain NRS, in 9/10 SF-36 scales, 4/10 MPI scales, 1/2 HADS scales, and 1/10 CSQ scales.

The correlation coefficients of the baseline scores and raw score differences, i.e. absolute effects (Table 2, 2^nd from last and last columns) for the scales under comparison showed moderately till highly (correlation coefficient > 0.70) overlapping constructs. The highest values reflecting good convergence were found for the affective health comparisons with values of up to 0.73 and for pain with values of up to 0.76. In the function and the physical role interference comparisons, only the comparison of MPI interference with SF-36 social functioning showed moderate correlations (up 0.50). The SF-36 physical functioning did not correlate with the MPI activity showing divergent/discriminant constructs.