Table 2 Studies attempting to quantify PRECIS dimensions since 2010
# | Study | Author | Year | Ref. | Context of use and development | Numeric scale | Delphi/iterative process | Conclusions | Limitations |
|---|---|---|---|---|---|---|---|---|---|
1 | The Pragmatic–Explanatory Continuum Indicator Summary (PRECIS) instrument was useful for refining a randomized trial design: experiences from an investigative team | Riddle et al. | 2010 | [17] | ·Prospective use of the tool to modify study design in 3-arm study of pain coping in patients scheduled to have a knee replacement 1-day meeting of 7 authors/investigators: 1) Pre-read Thorpe et al., 20092) Discuss goal of study3) Discuss criteria4) Initial blinded rating5) Ideal blinded rating6) 3rd rating to see if consensus was reached | · Yes: 4-cm line to be marked independently on paper in person in 5 min followed by discussion · Scores measured by a ruler | Yes | · Modification of study· Closer consensus and more explanatory in design | · 4-cm scale using VAS; not applicable to online ratings |
2 | Alternative approaches to tuberculosis treatment evaluation: the role of pragmatic trials | Bratton et al. | 2011 | [18] | · Retrospective· 2 reviewers rated 3 published studies on TB treatment and discussed how to rate them on each dimension | · No, but modification to the dimensions made (practitioner expertise) combined and a new spoke for blindness inserted | · Yes, but not described in detail | · Pragmatic trials in TB might lessen time to implementation in real-world settings | · No metric at all, simply subjective |
3 | Pragmatic vs explanatory trials: the Pragmascope tool to help measure differences in protocols of mental health RCTs | Tosh et al. | 2011 | [19] | · Adapt (PRECIS) to assist researchers during protocol stage of RCTs in mental health (the Pragmascope tool)· Retrospective· 3 reviewers · Cochrane Schizophrenia Group Trials Register and Medline (November 2010) for references of RCT protocols. Chose a random sample of 10 protocols dealing with schizophrenia, depression, post-traumatic stress disorders, and psychiatric rehabilitation | · 1 to 5; 0 for missing information· Total score, 0 to 50· 0 to 30, explanatory; 31- 39, intermediate; ≥35, explanatory· However: Figure 1 (see main text) demonstrates an explanatory study investigating whether the experimental intervention will work in ideal circumstances (total score 0–15) and a more pragmatic study focusing mostly on whether, in routine practice, an intervention has a meaningful effect (total score >35)· A total score between 16 and 35 suggests an interim where trial design balances pragmatic and explanatory domains | · Not described | · Useful tool given high inter-rater reliability | · Scoring not clear; neither why the midpoint of 25 was not chosen as a balanced trial· Does the total score perpetuate the dichotomous classification of studies? |
4 | Applying the PRECIS criteria to describe 3 effectiveness trials of weight loss in obese patients with comorbid conditions. | Glasgow et al. | 2011 | [20] | · Practice-based Opportunities for WEight Reduction (POWER) trials (3 studies)· Reduce weight in primary care in those with one CVD risk factor. Studies were ongoing and in the field· 9 reviewers scored each protocol in a 4-step process. 1) Read article and review webinar by Sackett2) Score using 0 to 4 on the dimension· 8 additional ratings; representation of participants and settings, inclusion of cost estimates, reporting on context and level of engagement with the primary care practices | · 0 to 4 on the dimension with total composite scores also | · Not formally described, but does mention frequent calls to gain consistency in the interpretation of the dimensions | · Requires discussion and training to clarify each criteria | · ? Rater bias towards pragmatic· Scale difficult to interpret, but need cognitive testing as part of a qualitative study· Hard to find a reliability index |
5 | Pragmatic vs. explanatory: an adaptation of the PRECIS tool helps to judge the applicability of systematic reviews for daily practice | Koppenaal et al. | 2011 | [21] | · Improve lifestyle in general; should be implemented in general practice· Modified the PRECIS tool (called PRECIS Review tool [PR tool]) to grade individual trials and systematic reviews of trials· This should help policy makers, clinicians, researchers, and guideline developers to judge the applicability of individual trials and systematic reviews· 2 systematic reviews | · 1 to 5 rating· Individual studies and the review itself were scored· Abandoned VAS with 0 to 10 due to the arbitrary distinctions between consecutive scores· Used a Likert-type scale of 1 to 5, with concurrent %· If 3 dimensions unscorable, a randomized controlled trial was not to be used· 1 to 5 less arbitrary than 0 to 10 for this type of review | · 2 independent scores per study and then discussion to reach consensus· If unable, then consultation with third rater | ·Useful tool for reviewers and policy makers and trialists· Ability to detect heterogeneity in studies included in systematic reviews | · Missing information more likely for a dimension to be rated as pragmatic· Applicability to setting and context needs to considered by policy makers even if a trial is considered pragmatic· Equal weighting for each dimension |