Table 1 Characteristics of 387 systematics reviews in the context of missing outcome data
Characteristic | Levels | Total | 2009 (n = 21) | 2010 (n = 18) | 2011 (n = 48) | 2012 (n = 50) | 2013 (n = 65) | 2014 (n = 92) | 2015 (n = 68) | 2016 (n = 14) | 2017 (n = 11) | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
n | % | |||||||||||||||||||||
Addressing and handling missing outcome data | ||||||||||||||||||||||
Study protocol | Registered | 24 | 6.2 | 0 | 0.0 | 0 | 0.0 | 1 | 4.2 | 1 | 4.2 | 6 | 25.0 | 8 | 33.3 | 8 | 33.3 | 0 | 0.0 | 0 | 0.0 | |
Not registered but published | 14a | 3.6 | 1 | 7.1 | 0 | 0.0 | 2 | 14.3 | 2 | 14.3 | 6 | 42.9 | 1 | 7.1 | 2 | 14.3 | 0 | 0.0 | 0 | 0.0 | ||
Mentioned but not available | 66 | 17.1 | 4 | 6.1 | 3 | 4.5 | 12 | 18.2 | 6 | 9.1 | 10 | 15.2 | 17 | 25.8 | 9 | 13.6 | 2 | 3.0 | 3 | 4.5 | ||
Explicitly mentioned that there is no protocol | 4 | 1.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 25.0 | 0 | 0.0 | 3 | 75.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
Protocol not mentioned | 279 | 72.1 | 16 | 5.7 | 15 | 5.4 | 33 | 11.8 | 40 | 14.3 | 43 | 15.4 | 63 | 22.6 | 49 | 17.6 | 12 | 4.3 | 8 | 2.9 | ||
If protocol is available (37 SRs), the MOD strategy was determined already in the protocolb | Yes, using MOD as secondary outcome | 5 | 13.5 | 1 | 0.2 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 3 | 0.6 | 1 | 0.2 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | |
Yes, in primary analysis using ITT with clarifications | 7 | 18.9 | 1 | 14.3 | 0 | 0.0 | 1 | 14.3 | 1 | 14.3 | 2 | 28.5 | 1 | 14.3 | 1 | 14.3 | 0 | 0.0 | 0 | 0.0 | ||
Yes, in primary analysis using ITT without clarifications | 2 | 5.4 | 0 | 0.0 | 0 | 0.0 | 1 | 50.0 | 0 | 0.0 | 1 | 50.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
Yes, in primary analysis by excluding trials with MOD | 1 | 2.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 100 | 0 | 0.0 | 0 | 0.0 | ||
Yes, in primary analysis by excluding participants with MOD | 1 | 2.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 100 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
No MOD strategy was determined | 23 | 62.2 | 0 | 0.0 | 0 | 0.0 | 1 | 4.3 | 1 | 4.3 | 7 | 30.4 | 7 | 30.4 | 7 | 30.4 | 0 | 0.0 | 0 | 0.0 | ||
If protocol is available with a MOD strategy (14 SRs), the strategy defined in the protocol differed from that applied in the review | No, the authors adhered to the protocol | 12 | 85.7 | 0 | 0.0 | 0 | 0.0 | 2 | 16.7 | 1 | 8.3 | 1 | 8.3 | 4 | 33.3 | 4 | 33.3 | 0 | 0.0 | 0 | 0.0 | |
MOD were not addressed eventually | 2 | 14.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 50.0 | 1 | 50.0 | 0 | 0.0 | 0 | 0.0 | ||
The authors explained (in the protocol or review) what they considered as MOD | Yes, an explicit definition is provided | 63 | 16.3 | 4 | 6.3 | 7 | 11.1 | 13 | 20.6 | 4 | 6.3 | 12 | 19.0 | 11 | 17.5 | 8 | 12.7 | 3 | 4.8 | 1 | 1.6 | |
No explanation is provided | 324 | 83.7 | 17 | 5.2 | 11 | 3.4 | 35 | 10.8 | 46 | 14.2 | 53 | 16.4 | 81 | 25.0 | 60 | 18.5 | 11 | 3.4 | 10 | 3.1 | ||
The review explicitly reported whether LOCF was employed or not in the included trials | Yes, and they distinguished between LOCF and completely MOD | 5 | 1.3 | 0 | 0.0 | 1 | 20.0 | 1 | 20.0 | 0 | 0.0 | 2 | 40.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 20.0 | |
Yes, but they didn’t distinguish between LOCF and completely MOD | 18 | 4.6 | 0 | 0.0 | 1 | 5.6 | 4 | 22.2 | 2 | 11.1 | 3 | 16.7 | 6 | 33.3 | 1 | 5.6 | 0 | 0.0 | 1 | 5.6 | ||
No | 364 | 94.1 | 21 | 5.8 | 16 | 4.4 | 43 | 11.8 | 48 | 13.2 | 60 | 16.5 | 86 | 23.6 | 67 | 18.4 | 14 | 3.8 | 9 | 2.5 | ||
There is no evidence that MOD exist in the included trials for the primary outcomes | MOD are present in the network | 273 | 70.5 | 10 | 3.7 | 14 | 5.1 | 35 | 12.8 | 29 | 10.6 | 44 | 16.1 | 76 | 27.8 | 44 | 16.1 | 13 | 4.8 | 8 | 2.9 | |
Yes | 1 | 0.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 100 | 0 | 0.0 | 0 | 0.0 | ||
Nothing mentioned | 113 | 29.2 | 11 | 9.7 | 4 | 3.5 | 13 | 11.5 | 21 | 18.6 | 21 | 18.6 | 16 | 14.2 | 23 | 20.4 | 1 | 0.9 | 3 | 2.7 | ||
If the review explicitly reported the presence of MOD (273 SRs), the strategy described in the Methods section to address MOD isb | by excluding participants with MOD from the analyses | 4 | 1.5 | 0 | 0.0 | 0 | 0.0 | 1 | 25.0 | 1 | 25.0 | 0 | 0.0 | 1 | 25.0 | 0 | 0.0 | 0 | 0.0 | 1 | 25.0 | |
using ITT in the primary analysis with clarifications | 25 | 9.2 | 2 | 8.0 | 1 | 4.0 | 3 | 12.0 | 3 | 12.0 | 4 | 16.0 | 8 | 32.0 | 3 | 12.0 | 0 | 0.0 | 1 | 4.0 | ||
using ITT in the primary analysis without further clarifications | 84 | 30.8 | 1 | 1.2 | 1 | 1.2 | 10 | 11.9 | 10 | 11.9 | 20 | 23.8 | 23 | 27.4 | 14 | 16.7 | 3 | 3.6 | 2 | 2.3 | ||
using dropout as a secondary outcome | 61 | 22.3 | 4 | 6.6 | 4 | 6.6 | 12 | 19.7 | 4 | 6.6 | 14 | 22.9 | 11 | 18.0 | 7 | 11.5 | 3 | 4.9 | 2 | 3.2 | ||
using sensitivity analysis | by excluding studies based on a MOD rate threshold | 6 | 2.2 | 2 | 33.3 | 0 | 0.0 | 1 | 16.7 | 0 | 0.0 | 1 | 16.7 | 1 | 16.7 | 1 | 16.7 | 0 | 0.0 | 0 | 0.0 | |
by excluding participants with MOD | 6 | 2.2 | 0 | 0.0 | 1 | 16.7 | 1 | 16.7 | 1 | 16.7 | 0 | 0.0 | 2 | 33.3 | 1 | 16.7 | 0 | 0.0 | 0 | 0.0 | ||
other | 4 | 1.5 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 25.0 | 0 | 0.0 | 0 | 0.0 | 2 | 50.0 | 1 | 25.0 | 0 | 0.0 | ||
using subgroup analysis | 6 | 2.2 | 0 | 0.0 | 1 | 16.7 | 0 | 0.0 | 0 | 0.0 | 3 | 50.0 | 2 | 33.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
using meta-regression analysis | 3 | 1.1 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 33.3 | 2 | 66.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
Not mentioned | 120 | 44.0 | 4 | 3.3 | 9 | 7.5 | 13 | 10.8 | 11 | 9.2 | 14 | 11.7 | 34 | 28.3 | 24 | 20.0 | 7 | 5.9 | 4 | 3.3 | ||
In case of ‘intention-to-treat analysis’ (109 SRs), did the authors extract data as reported in the trials or they applied ITT de novo? | Data extracted as reported in the trials | 46 | 42.2 | 0 | 0.0 | 1 | 2.2 | 3 | 6.5 | 3 | 6.5 | 11 | 23.9 | 15 | 32.6 | 8 | 17.4 | 3 | 6.5 | 2 | 4.3 | |
Intention-to-treat analysis de novo | 31 | 28.4 | 1 | 3.2 | 1 | 3.2 | 6 | 19.4 | 3 | 9.7 | 6 | 19.4 | 11 | 35.5 | 2 | 6.5 | 0 | 0.0 | 1 | 3.2 | ||
Combination of the aforementioned | 13 | 12.0 | 1 | 7.7 | 0 | 0.0 | 3 | 23.1 | 4 | 30.8 | 1 | 7.7 | 1 | 7.7 | 3 | 23.1 | 0 | 0.0 | 0 | 0.0 | ||
Unclear | 19 | 17.4 | 1 | 5.3 | 0 | 0.0 | 1 | 5.3 | 3 | 15.8 | 6 | 31.6 | 4 | 21.1 | 4 | 21.1 | 0 | 0.0 | 0 | 0.0 | ||
Bias due to MOD was evaluated | Yes, using a specific tool | 198c | 51.2 | 6 | 3.0 | 8 | 4.0 | 21 | 10.6 | 15 | 7.6 | 35 | 17.7 | 52 | 26.3 | 43 | 21.7 | 10 | 5.1 | 8 | 4.0 | |
Yes, probably but results are not displayed | 72 | 18.6 | 5 | 6.9 | 3 | 4.2 | 9 | 12.5 | 11 | 15.3 | 14 | 19.4 | 20 | 27.8 | 9 | 12.5 | 0 | 0.0 | 1 | 1.4 | ||
No, but other domains were evaluated | 25 | 6.5 | 2 | 8.0 | 1 | 4.0 | 4 | 16.0 | 4 | 16.0 | 4 | 16.0 | 6 | 24.0 | 3 | 12.0 | 0 | 0.0 | 1 | 4.0 | ||
No evaluation of risk of bias | 92 | 23.8 | 8 | 8.7 | 6 | 6.5 | 14 | 15.2 | 20 | 21.7 | 12 | 13.0 | 14 | 15.2 | 13 | 14.1 | 4 | 4.3 | 1 | 1.1 | ||
Bias due to MOD was evaluated explicitly | No explicit evaluation | 181 | 46.8 | 6 | 3.3 | 8 | 4.4 | 21 | 11.6 | 15 | 8.3 | 33 | 18.2 | 46 | 25.4 | 38 | 21.0 | 8 | 4.4 | 6 | 3.3 | |
With justification of judgments | 18c | 4.6 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 3 | 17.6 | 6 | 35.3 | 4 | 23.5 | 2 | 11.8 | 2 | 11.8 | ||
Results on bias due to MOD are not displayed | 15 | 3.9 | 3 | 20.0 | 2 | 13.3 | 2 | 13.3 | 5 | 33.3 | 1 | 6.7 | 0 | 0.0 | 2 | 13.3 | 0 | 0.0 | 0 | 0.0 | ||
No evaluation of bias due to MOD | 25 | 6.5 | 2 | 8.0 | 1 | 4.0 | 4 | 16.0 | 4 | 16.0 | 4 | 16.0 | 6 | 24.0 | 3 | 12.0 | 0 | 0.0 | 1 | 4.0 | ||
Only an overall score is provided for each trial | 37 | 9.6 | 1 | 2.7 | 1 | 2.7 | 6 | 16.2 | 6 | 16.2 | 7 | 18.9 | 12 | 32.4 | 4 | 10.8 | 0 | 0.0 | 0 | 0.0 | ||
Results on the risk of bias evaluation are not presented | 19 | 4.9 | 1 | 5.3 | 0 | 0.0 | 1 | 5.3 | 0 | 0.0 | 5 | 26.3 | 8 | 42.1 | 3 | 15.8 | 0 | 0.0 | 1 | 5.3 | ||
No evaluation of risk of bias | 92 | 23.8 | 8 | 8.7 | 6 | 6.5 | 14 | 15.2 | 20 | 21.7 | 12 | 13.0 | 14 | 15.2 | 13 | 14.1 | 4 | 4.3 | 1 | 1.1 | ||
Acknowledging implications of missing outcome data | ||||||||||||||||||||||
Among the reviews with MOD (273 SRs), the interpreted results accounted for MOD | Yes | 88 | 32.2 | 4 | 4.5 | 7 | 8.0 | 16 | 18.2 | 8 | 9.1 | 17 | 19.3 | 20 | 22.7 | 10 | 11.4 | 4 | 4.5 | 2 | 2.3 | |
No | 185 | 67.8 | 6 | 3.2 | 7 | 3.8 | 19 | 10.3 | 21 | 11.4 | 27 | 14.6 | 56 | 30.3 | 34 | 18.4 | 9 | 4.9 | 6 | 3.2 | ||
If the interpreted results accounted for MOD (88 SRs), MOD implications were reported inb | Abstract | 46 | 52.3 | 3 | 6.5 | 2 | 4.3 | 8 | 17.4 | 3 | 6.5 | 9 | 19.6 | 11 | 23.9 | 5 | 11.0 | 3 | 6.5 | 2 | 4.3 | |
Results | 26 | 29.5 | 1 | 3.8 | 2 | 7.7 | 4 | 15.4 | 2 | 7.7 | 4 | 15.4 | 8 | 30.8 | 4 | 15.4 | 1 | 3.8 | 0 | 0.0 | ||
Discussion | 74 | 84.1 | 4 | 5.4 | 5 | 6.8 | 13 | 17.6 | 6 | 8.1 | 16 | 21.6 | 18 | 24.3 | 7 | 9.5 | 3 | 4.0 | 2 | 2.7 | ||
Conclusions | 11 | 12.5 | 0 | 0.0 | 1 | 9.1 | 2 | 18.2 | 1 | 9.1 | 2 | 18.2 | 2 | 18.2 | 1 | 9.1 | 0 | 0.0 | 2 | 18.2 | ||
If the interpreted results accounted for MOD (88 SRs), they were discussed in the context of which NMA componentsb | NMA treatment effects | 84 | 95.5 | 4 | 4.8 | 7 | 8.3 | 15 | 17.8 | 7 | 8.3 | 16 | 19.0 | 20 | 23.8 | 10 | 11.9 | 4 | 4.8 | 1 | 1.2 | |
Intervention ranking | 13 | 14.8 | 1 | 7.7 | 1 | 7.7 | 2 | 15.4 | 1 | 7.7 | 3 | 23.0 | 1 | 7.7 | 1 | 7.7 | 2 | 15.4 | 1 | 7.7 | ||
Heterogeneity | 7 | 8.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 2 | 28.6 | 2 | 28.6 | 2 | 28.6 | 1 | 14.2 | 0 | 0.0 | 0 | 0.0 | ||
Evidence consistency | 3 | 3.4 | 0 | 0.0 | 1 | 33.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 2 | 66.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
What information the authors used to discuss the implications (88 SRs)b | Judgments from the risk of bias assessment | 10 | 11.4 | 1 | 10.0 | 1 | 10.0 | 1 | 10.0 | 1 | 10.0 | 1 | 10.0 | 5 | 50.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | |
The comments on the quality of evidence in SoF table | 1 | 1.1 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 100 | ||
Sensitivity analysis results | 16 | 18.2 | 1 | 6.2 | 1 | 6.2 | 2 | 12.5 | 2 | 12.5 | 1 | 6.2 | 4 | 25.0 | 4 | 25.0 | 1 | 6.2 | 0 | 0.0 | ||
Subgroup analysis on a dropout-relevant characteristic | 4 | 4.5 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 3 | 75.0 | 1 | 25.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
Meta-regression analysis using dropout as covariate | 3 | 3.4 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 33.3 | 2 | 66.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
NMA results on dropout (as a secondary outcome) | 58 | 65.9 | 4 | 6.9 | 3 | 5.2 | 11 | 19.0 | 4 | 6.9 | 14 | 24.1 | 10 | 17.2 | 7 | 12.1 | 3 | 5.2 | 2 | 3.4 | ||
Dropout prevalence | 11 | 12.5 | 0 | 0.0 | 2 | 18.2 | 4 | 36.4 | 1 | 9.1 | 1 | 9.1 | 2 | 18.2 | 1 | 9.1 | 0 | 0.0 | 0 | 0.0 | ||
Reasons for dropout | 1 | 1.1 | 0 | 0.0 | 1 | 100 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
The strategy used to handle MOD in primary analysis | 2 | 2.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 50.0 | 1 | 50.0 | 0 | 0.0 | 0 | 0.0 | ||