Skip to main content

Table 2 Suggestions for a more valid assessment of intervention effects in systematic reviews

From: Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods

Step 1 Calculate and report the P-values and the 95% confidence intervals from all fixed-effect and random-effects meta-analyses. The most conservative result should be the main result.
Step 2 Explore the reasons behind substantial statistical heterogeneity by performing subgroup analyses and sensitivity analyses (see step 6).
Step 3 Adjust the thresholds for significance (P-values and the confidence intervals from the meta-analyses and the risks of type I error in the trial sequential analysis) according to the number of primary outcome comparisons.
Step 4 Calculate and report a realistic diversity-adjusted required information size and analyse all of the outcomes with trial sequential analysis. Report if the trial sequential monitoring boundaries for benefit, harm, or futility are crossed.
Step 5 Calculate and report Bayes factor for the primary outcome/s based on the anticipated intervention effect used to estimate the required information size (http://www.ctu.dk/tools-and-links/bayes-factor-calculation.aspx). A Bayes factor less than 0.1 (a ten-fold higher likelihood of compatibility with the alternative hypothesis than with the null hypothesis) may be chosen as threshold for significance.
Step 6 Use subgroup analysis and sensitivity analyses to assess the potential impact of systematic errors (‘bias’).
Step 7 Assess the risk of publication bias.
Step 8 Assess clinical significance of the review results if the prior seven steps have shown statistically significant results.
  1. All of these aspects should be prospectively planned and published in the protocol for the systematic review before the literature search begins.