Table 3 Interrater agreement for AMSTAR
Item | Gwet’s AC1 | SEM | 95% CI |
|---|---|---|---|
1. Was an ‘a priori’ design provided | 0.90 | 0.04 | 0.82–0.99 |
2. Was there duplicate study selection and data extraction | 0.47 | 0.09 | 0.29–0.66 |
3. Was a comprehensive literature search performed | 0.92 | 0.03 | 0.85–0.98 |
4. Was the status of publication (i.e. grey literature) used as an inclusion criterion | 0.03 | 0.11 | −0.19-0.24 |
5. Was a list of studies (included and excluded) provided | 0.63 | 0.08 | 0.47–0.79 |
6. Were the characteristics of the included studies provided | −0.09 | 0.10 | −0.30-0.12 |
7. Was the scientific quality of the included studies assessed and documented | 0.89 | 0.04 | 0.81–0.97 |
8. Was the scientific quality of the included studies used appropriately in formulating conclusions | 0.68 | 0.08 | 0.53–0.83 |
9. Were the methods used to combine the findings of studies appropriate | 0.88 | 0.04 | 0.79–0.96 |
10. Was the likelihood of publication bias assessed | 0.76 | 0.07 | 0.62–0.89 |
11. Was the conflict of interest included | 0.80 | 0.05 | 0.70–0.91 |
Overall agreement (mean score of 11 items) | 0.62 | 0.10 | 0.39–0.86 |