Table 1 Quality assessment tools with published validity and reliability data – information reported by the scale developers
| Â | Instrument Description | Item Generation | Validity | Inter-Rater Reliability | Application in Study |
|---|---|---|---|---|---|
Jadad et al (22) | number of items: 3 or 6 scoring for 3 item: 0, 1, or 2 for two items 0 or 1 for 1 item range 0 to 5 scoring for 6 item: 0, 1, or 2 for two items 0 or 1 for 4 items range 0 to 8 cut offs not specified for either version | consensus among 6 judges (pain management &/or instrument development) pretested with 3 raters on 13 clinical trial reports | 3 groups of pain studies were identified a priori: • studies rated as excellent by experts (n = 7); overall score = 3.4 • studies rated as poor by experts (n = 6); overall score = 0.7 • a random selection of RCTs from a MEDLINE search (n = 23); overall score = 2.7 | ICC = 0.66 (95% CI: 0.53, 0.79) 14 raters | 3-item and 6-item versions in their entirety |
Sindhu et al (24) | number of items: 53 scoring: items weighted from 1 to 10 range 0 – 100 cut offs not specified | 8-member Delphi panel (clinicians and methodologists) | Compared with the Chalmers scale for five studies of non-pharmacologic nursing interventions for pain management; r = 0.94 (Pearson) 11 studies considered for a published meta-analysis on hypertension were evaluated: • 9 included studies scored 73–93%; • 2 rejected studies scored below 70% | ICC = 0.93 2 raters | 12 of 53 items eliminated because of difficulty applying to cancer studies 41 items related to 12 domains range 0 – 79.5 |
Downs & Black (25) | number of items: 27 scoring 0 or 1 for 25 items 1 or 2 for 1 item 0 to 5 for 1 item cut offs not specified | based on epidemiological principles, reviews of study designs and existing checklists pilot tested by 2 epidemiologists rating 20 studies | Compared with the Standards of Reporting Trials Group checklist scale for 10 RCTs on surgery for stress incontinence; r = 0.90 (Spearman) | Spearman r = 0.75 2 raters | used 13 internal validity items only items not related to our definition of quality eliminated (10 reporting items, 3 external validity items, 1 power item) range 1 to 13 |