From: Overview of data-synthesis in systematic reviews of studies on outcome prediction models
 |  |  |  | n = 50 reviews | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Item: | Description of item: | Â | Â | Yes | No | Unclear | Not applicable | ||||
 |  |  |  | N | % | N | % | N | % | N | % |
Section 1: Information about the objective and design of the reviews | Â | Â | Â | Â | Â | Â | Â | Â | |||
1. | Type of primary studies included | n = 50 (%) |  |  |  |  |  |  |  |  | |
 |  Only outcome prediction models | 14 | (28.0) |  |  |  |  |  |  |  |  |
 |  Combination of prognostic factor & outcome prediction studies | 3 | (6.0) |  |  |  |  |  |  |  |  |
 |  Unclear | 33 | (66.0) |  |  |  |  |  |  |  |  |
2. | Is the outcome of interest clearly described? | Â | Â | 47 | (94.0) | 1 | (2.0) | 2 | (4.0) | Â | Â |
3. | Is information about quality assessment provided? | Â | Â | 36 | (72.0) | 14 | (28.0) | Â | Â | Â | Â |
3a. | Method used | Â | Â | Â | Â | Â | Â | Â | Â | Â | Â |
 |  Methodological criteria list | 3 | (6.0) |  |  |  |  |  |  |  |  |
 |  Individual items | 2 | (4.0) |  |  |  |  |  |  |  |  |
 |  Not applicable | 14 | (28.0) |  |  |  |  |  |  |  |  |
 |  Methodological criteria & study design | 31 | (62.0) |  |  |  |  |  |  |  |  |
4. | Was study quality accounted for | Â | Â | 21 | (42.0) | 13 | (26.0) | 2 | (4.0) | 14 | (28.0) |
4a. | Method used *# | n = 23 (%) |  |  |  |  |  |  |  |  | |
 |  Exclusion of poor quality studies (cut-off score used) | 3 | (13.0) |  |  |  |  |  |  |  |  |
 |  Sensitivity analysis based on total quality score | 5 | (21.7) |  |  |  |  |  |  |  |  |
 |  Levels of evidence | 12 | 52.2) |  |  |  |  |  |  |  |  |
 |  Subgroup analysis | 7 | (30.4) |  |  |  |  |  |  |  |  |
 |  Study findings weighted for quality | 3 | (13.0) |  |  |  |  |  |  |  |  |
 |  Other | 2 | (8.7) |  |  |  |  |  |  |  |  |
Section 2: Information about the design and results of the primary studies | Â | Â | Â | Â | Â | Â | Â | Â | |||
5. | Outcomes clearly described | 36 | (72.0) | 20 | (20.0) | 4 | (8.0) | Â | Â | ||
6. | Statistical methods used for variable selection described | 2 | (4.0) | 46 | (92.0) | 2 | (4.0) | Â | Â | ||
7. | Treatments described | 6 | (12.0) | 37 | (74.0) | 7 | (14.0) | Â | Â | ||
8. | Univariable point estimates for all the variables of the primary studies are provided | 5 | (10.0) | 42 | (84.0) | 3 | (6.0) | Â | Â | ||
8a. | Univariable estimates for dispersion for all the variables of the primary studies are provided | 5 | (10.0) | 42 | (84.0) | 3 | (6.0) | Â | Â | ||
9. | All variables (starting predictors) used to develop a model are described | 4 | (8.0) | 36 | (72.0) | 10 | (20.0) | Â | Â | ||
10. | Multivariable point estimates for each predictor in the final outcome prediction model are provided | 11 | (22.0) | 33 | (66.0) | 4 | (8.0) | 2 | (4.0) | ||
10a. | Multivariable estimate of dispersion provided for each predictor in the final outcome prediction model | 11 | (22.0) | 33 | (66.0) | 4 | (8.0) | 2 | (4.0) | ||
11. | Model performance is assessed and described | 7 | (14.0) | 38 | (76.0) | 2 | (4.0) | 3 | (6.0) | ||
12. | number of events per variable is described | 4 | (8.0) | 44 | (88.0) | 2 | (4.0) | Â | Â | ||
Section 3: Data-analysis and synthesis in the reviews | Â | Â | Â | Â | Â | Â | Â | Â | |||
13. | Heterogeneity between studies described | 45 | (90.0) | 4 | (8.0) | 1 | (2.0) | Â | Â | ||
14. | Qualitative data-synthesis presented | 49 | (98.0) | 1 | (2.0) | Â | Â | Â | Â | ||
14a. | Method used | n = 49 (%) |  |  |  |  |  |  |  |  | |
 |  Statistical significance | 22 | (44.9) |  |  |  |  |  |  |  |  |
 |  Consistency of findings | 7 | (14.3) |  |  |  |  |  |  |  |  |
 |  Consistency of findings & statistical significance | 6 | (12.2) |  |  |  |  |  |  |  |  |
 |  Available method of defining levels of evidence | 3 | (6.1) |  |  |  |  |  |  |  |  |
 |  Consistency of findings & levels of evidence | 3 | (6.1) |  |  |  |  |  |  |  |  |
 |  other combinations | 8 | (16.3) |  |  |  |  |  |  |  |  |
15. | Quantitative analysis performed | Â | Â | 10 | (20.0) | 40 | (80.0) | Â | Â | Â | Â |
15a. | Method used | n = 10 (%) |  |  |  |  |  |  |  |  | |
 |  Random effects model | 4 | (40.0) |  |  |  |  |  |  |  |  |
 |  Fixed effects model | 1 | (10.0) |  |  |  |  |  |  |  |  |
 |  Random & Fixed effects model | 3 | (30.0) |  |  |  |  |  |  |  |  |
 |  Other | 2 | (20.0) |  |  |  |  |  |  |  |  |
 |  |  |  | n = 10 reviews | |||||||
15b. | Statistical heterogeneity assessed | Â | Â | 4 | (40.0) | 6 | (60.0) | Â | Â | Â | Â |
15c. | Method used to assess statistical heterogeneity | n = 4 (%) |  |  |  |  |  |  |  |  | |
 |  I2 | 2 | (50.0) |  |  |  |  |  |  |  |  |
 |  I2 & Chi2 | 1 | (25.0) |  |  |  |  |  |  |  |  |
 |  Other | 1 | (25.0) |  |  |  |  |  |  |  |  |
 |  |  |  | n = 50 reviews | |||||||
16. | Graphic presentation of results provided | Â | Â | 8 | (16.0) | 42 | (84.0) | Â | Â | Â | Â |
16a. | Method used | n = 8 (%) |  |  |  |  |  |  |  |  | |
 |  Forest plot | 6 | (75.0) |  |  |  |  |  |  |  |  |
 |  Forest plot & scatter plot | 1 | (12.5) |  |  |  |  |  |  |  |  |
 |  Barplot | 1 | (12.5) |  |  |  |  |  |  |  |  |
17. | Sensitivity analysis performed | Â | Â | 6 | (12.0) | 43 | (86.0) | 1 | (2.0) | Â | Â |
17a. | Method used | n = 6 (%) |  |  |  |  |  |  |  |  | |
 |  Different cut-offs for study quality | 3 | (50.0) |  |  |  |  |  |  |  |  |
 |  Methodological criteria | 1 | (16.7) |  |  |  |  |  |  |  |  |
 |  Methodological criteria & weights for quality | 1 | (16.7) |  |  |  |  |  |  |  |  |
 |  Including other (excluded) cohorts | 1 | (16.7) |  |  |  |  |  |  |  |  |