Skip to main content

Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Table 1 Categories and elements of systematic review definition found in health care literature; percentage calculated from 226 sources of information that had a SR definition, or inclusion/exclusion criteria that could be used for extracting individual elements of SR definition

From: Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks

Category Element of definition N (%)
Self-identified as a systematic review Manuscript that identifies itself as a systematic review in title, abstract or in methods 30 (13)
Indexing Indexed as SR 1 (0.4)
Aim/research question Specific research question 66 (29)
Clearly stated set of objectives 12 (5.3)
Clearly formulated research question 11 (4.8)
Focused research question 3 (1.3)
Reported research question 2 (0.9)
Clinical question including participants, interventions, controls, outcomes and study design (PICOS) 2 (0.9)
Explicit clinical question 1 (0.4)
Clearly stated topic of review 1 (0.4)
Explicitly reported pre-defined objectives 1 (0.4)
Stated goal implied a critical and comprehensive intent 1 (0.4)
Clear statement of the topic 1 (0.4)
Defined clinical topic 1 (0.4)
Explicit statement of questions being addressed 1 (0.4)
Overall methods Systematic methods 22 (9.7)
Explicit methods 21 (9.2)
Systematic method to minimize risk of bias 9 (4)
Systematic approach, in an attempt to minimize biases and random errors, documented in the Materials and Methods section 8 (3.5)
Explicit method to minimize risk of bias 7 (3.1)
Reproducible methods 5 (2.2)
Using a systematic approach 5 (2.2)
Methods described in explicit detail 4 (1.8)
Well-defined methods 2 (0.9)
Overall methods defined study as systematic review 1 (0.4)
Overall Conduct defined study as a systematic review 1 (0.4)
Systematic review methodology on closer inspection of the methods section 1 (0.4)
Specific methods 1 (0.4)
Repeatable methods 1 (0.4)
Rigorous methods 1 (0.4)
Different components of the review process documented in the ‘methods section’ 1 (0.4)
Using methods to provide more reliable findings 1 (0.4)
Using methods from which conclusions can be drawn 1 (0.4)
Using methods based on which decisions can be made 1 (0.4)
Exhaustive review of the literature 1 (0.4)
Systematic approach 1 (0.4)
Search Systematic search 29 (13)
Reported search strategy 13 (5.8)
Comprehensive search strategy 12 (5.3)
Searched at least two databases/sources 10 (4.4)
Exact search criteria reported 9 (4.0)
Searched at least one database 9 (3.9)
Reported search methods 7 (3.1)
Attempt to collate all empirical evidence 7 (3.1)
Reported all information sources 6 (2.6)
Transparent search strategy 6 (2.6)
Detailed and comprehensive search strategy (as identified by: naming of databases and years of searching and example or actual terms) 4 (1.8)
Detailed and specific search strategy with key-words that enabled reproduction of the literature search 4 (1.8)
Names of databases reported 4 (1.8)
Explicit search criteria that are available to review 3 (1.3)
Description of data sources and search dates 2 (0.4)
Keywords searched 2 (0.9)
Detailed search of the literature for relevant studies 2 (0.9)
Explicit description of search strategy 2 (0.9)
Adequate searching methods 2 (0.9)
Replicable search method 2 (0.9)
Reported search sources 1 (0.4)
Description of sources 1 (0.4)
Reported details of databases searched 1 (0.4)
Reported dates of search 1 (0.4)
Included relevant search strategy 1 (0.4)
Adequate search strategy 1 (0.4)
Appropriate search strategy 1 (0.4)
Detailed search strategy 1 (0.4)
Non-selective search strategy 1 (0.4)
Explicit search strategy 1 (0.4)
Prescriptive search strategy 1 (0.4)
Reproducible search strategy 1 (0.4)
Rigorous search process 1 (0.4)
Explicitly reported search strategy details 1 (0.4)
Thorough search of evidence 1 (0.4)
Comprehensive search of evidence 1 (0.4)
Reported search processes 1 (0.4)
Extensive use of search string combinations 1 (0.4)
Description of evidence retrieval methods 1 (0.4)
Explicit and organized approach to searching 1 (0.4)
Attempt to search all empirical evidence 1 (0.4)
Adequately attempt to retrieve all relevant data 1 (0.4)
Review trying to collect all available evidence 1 (0.4)
Structured search of bibliographic and other databases 1 (0.4)
Searched at least Medline 1 (0.4)
Searched at least two databases (of which one is Medline) 1 (0.4)
Identification of studies Explicit methods to identify relevant research 14 (6.2)
Systematic methods of identification of studies 10 (4.4)
Attempt to identify all empirical evidence 6 (2.6)
Reported methods for identification of studies 2 (0.9)
Transparent procedure to find relevant research 2 (0.9)
Formal process of identifying literature 1 (0.4)
Selection of studies Explicit methods to select relevant research 14 (6.2)
Systematic methods of selection of studies 13 (5.8)
Reported methods for selection of studies 6 (2.6)
Transparent selection of studies 2 (0.9)
Reproducible selection of studies 4 (1.8)
Reproducible approach for selecting the studies 1 (0.4)
Clear description of selection criteria 1 (0.4)
Clear study selection criteria 1 (0.4)
Relevant study selection criteria 1 (0.4)
Detailed description of the studies’ selection process (number of articles included and excluded in each step) 1 (0.4)
Study eligibility Reported inclusion and exclusion criteria 31 (14)
Pre-defined/pre-specified eligibility criteria 20 (8.8)
Outcome defined using a validated tool or diagnostic criteria 13 (5.8)
Only Cochrane systematic reviews 12 (5.3)
Reported inclusion criteria 6 (2.6)
Explicitly reported inclusion and exclusion criteria 6 (2.6)
Articles that meet PRISMA definition of a systematic review 5 (2.2)
Definitions of the population(s), intervention(s), comparator(s) and outcome(s) of interest 2 (0.9)
Inclusion/exclusion criteria that are relevant in terms of the PICO framework 3 (1.3)
Reviews published in Database of Reviews of Effects (DARE) 2 (0.9)
Reviews were judged to be systematic if they synthesized peer reviewed articles 1 (0.4)
Studies meeting minimum methodological standards 1 (0.4)
Reference to study designs 1 (0.4)
Data extraction Systematic data collection 12 (5.3)
Systematic methods to extract data 4 (1.8)
Explicit methods to collect data 3 (1.3)
Data extraction by 2 independent reviewers 2 (0.9)
Reported data abstraction from trials 2 (0.9)
Independent data extraction 1 (0.4)
Explicit approach to extracting 1 (0.4)
Organized approach to extracting 1 (0.4)
Explicit methods to extract data 1 (0.4)
Performed data extraction 1 (0.4)
Extracting the information from the studies following a priori protocol 1 (0.4)
Quality, bias, appraisal, validity Quality assessment of evidence 27 (12)
Critical appraisal of the studies 25 (11)
Risk of bias assessment 19 (8.4)
Systematic methods to critically appraise relevant research 13 (5.8)
Explicit methods to critically appraise relevant research 13 (5.8)
Reported validity assessment 11 (4.9)
Attempt to appraise all empirical evidence 6 (2.6)
Full assessment of methodological quality of included studies 5 (2.2)
Consideration of internal and external validity of the research 3 (1.3)
Provided sufficient details about individual included studies to enable assessment of quality by a reader 2 (0.9)
Reported at least one or more aspects of validity assessment of original studies 2 (0.9)
Transparent procedures to evaluate relevant research 2 (0.9)
Full report of methodological quality of included studies 1 (0.4)
Transparent process to minimize risk of bias 1 (0.4)
Explicit approach to critically evaluating studies 1 (0.4)
Organized approach to critically evaluating empirical literature 1 (0.4)
Systematic approach for assessing the studies 1 (0.4)
Reproducible approach for assessing the studies 1 (0.4)
Assessed methodological features of the included studies 1 (0.4)
Adequate methods to appraise included studies 1 (0.4)
Transparent methodological criteria are used to exclude papers that do not meet an explicit methodological benchmark 1 (0.4)
Evaluate the retrieved studies using prospectively defined methodological criteria 1 (0.4)
Analysis, synthesis Synthesis of results 34 (15)
Presence of meta-analysis 19 (10)
Systematic methods of analysis of studies 18 (8.0)
Explicit methods to analyze data 17 (7.5)
Systematic synthesis of findings 10 (4.4)
Quantitative synthesis 9 (4.0)
Synthesis of the included evidence, whether narrative or quantitative 7 (3.1)
Attempt to synthesize all empirical evidence 6 (2.6)
Systematic analysis of results 2 (0.9)
Unbiased synthesis of study findings 2 (0.9)
Transparent procedures to synthesize the results of relevant research 2 (0.9)
Analyze results appropriately 1 (0.4)
Systematic analysis 1 (0.4)
Plausible analysis of data 1 (0.4)
Plausible synthesis of data 1 (0.4)
Summary of results 1 (0.4)
Systematic analysis 1 (0.4)
Meta-analysis or best evidence synthesis 1 (0.4)
Formal analysis contained in the methods 1 (0.4)
Makes judgement about research question 1 (0.4)
Relying on statistical significance to make judgments about what works 1 (0.4)
Transparent process of interpretation of the findings of the studies included in the review 1 (0.4)
Rigorous conclusions about outcomes 1 (0.4)
Describing included studies Systematic presentation of characteristics of included studies 4 (1.8)
Systematic synthesis of characteristics of included studies 4 (1.8)
Clearly identified all included studies 2 (0.9)
Reported trial characteristics 1 (0.4)
Systematic presentation of main information 1 (0.4)
Described main characteristics of included studies 1 (0.4)
Adequate methods to describe included studies 1 (0.4)
Description of the number and nature of included studies 1 (0.4)
Description of the types of primary studies included 1 (0.4)
Accounted for identified studies 1 (0.4)
Reporting Used PRISMA or predecessor guidelines for reporting 3 (2)
Presented results appropriately 1 (0.4)
Systematic presentation of findings 1 (0.4)
Flow chart present 1 (0.4)
Reported level of evidence for their recommendations 1 (0.4)
Reported sufficient information to allow a level of evidence grading 1 (0.4)
Published in a journal conforming to PRISMA standards 1 (0.4)
A review that has methods and results section 1 (0.4)
Unclear “It was apparent in the text that a systematic review had been undertaken” 4 (1.8)
“Reviews were included if they were systematic” 1 (0.4)