Table 1 Categories and elements of systematic review definition found in health care literature; percentage calculated from 226 sources of information that had a SR definition, or inclusion/exclusion criteria that could be used for extracting individual elements of SR definition

Self-identified as a systematic review

Manuscript that identifies itself as a systematic review in title, abstract or in methods

Indexing

Indexed as SR

Aim/research question

Specific research question

Clearly stated set of objectives

Clearly formulated research question

Focused research question

Reported research question

Clinical question including participants, interventions, controls, outcomes and study design (PICOS)

Explicit clinical question

Clearly stated topic of review

Explicitly reported pre-defined objectives

Stated goal implied a critical and comprehensive intent

Clear statement of the topic

Defined clinical topic

Explicit statement of questions being addressed

Overall methods

Systematic methods

Explicit methods

Systematic method to minimize risk of bias

Systematic approach, in an attempt to minimize biases and random errors, documented in the Materials and Methods section

Explicit method to minimize risk of bias

Reproducible methods

Using a systematic approach

Methods described in explicit detail

Well-defined methods

Overall methods defined study as systematic review

Overall Conduct defined study as a systematic review

Systematic review methodology on closer inspection of the methods section

Specific methods

Repeatable methods

Rigorous methods

Different components of the review process documented in the ‘methods section’

Using methods to provide more reliable findings

Using methods from which conclusions can be drawn

Using methods based on which decisions can be made

Exhaustive review of the literature

Systematic approach

Search

Systematic search

Reported search strategy

Comprehensive search strategy

Searched at least two databases/sources

Exact search criteria reported

Searched at least one database

Reported search methods

Attempt to collate all empirical evidence

Reported all information sources

Transparent search strategy

Detailed and comprehensive search strategy (as identified by: naming of databases and years of searching and example or actual terms)

Detailed and specific search strategy with key-words that enabled reproduction of the literature search

Names of databases reported

Explicit search criteria that are available to review

Description of data sources and search dates

Keywords searched

Detailed search of the literature for relevant studies

Explicit description of search strategy

Adequate searching methods

Replicable search method

Reported search sources

Description of sources

Reported details of databases searched

Reported dates of search

Included relevant search strategy

Adequate search strategy

Appropriate search strategy

Detailed search strategy

Non-selective search strategy

Explicit search strategy

Prescriptive search strategy

Reproducible search strategy

Rigorous search process

Explicitly reported search strategy details

Thorough search of evidence

Comprehensive search of evidence

Reported search processes

Extensive use of search string combinations

Description of evidence retrieval methods

Explicit and organized approach to searching

Attempt to search all empirical evidence

Adequately attempt to retrieve all relevant data

Review trying to collect all available evidence

Structured search of bibliographic and other databases

Searched at least Medline

Searched at least two databases (of which one is Medline)

Identification of studies

Explicit methods to identify relevant research

Systematic methods of identification of studies

Attempt to identify all empirical evidence

Reported methods for identification of studies

Transparent procedure to find relevant research

Formal process of identifying literature

Selection of studies

Explicit methods to select relevant research

Systematic methods of selection of studies

Reported methods for selection of studies

Transparent selection of studies

Reproducible selection of studies

Reproducible approach for selecting the studies

Clear description of selection criteria

Clear study selection criteria

Relevant study selection criteria

Detailed description of the studies’ selection process (number of articles included and excluded in each step)

Study eligibility

Reported inclusion and exclusion criteria

Pre-defined/pre-specified eligibility criteria

Outcome defined using a validated tool or diagnostic criteria

Only Cochrane systematic reviews

Reported inclusion criteria

Explicitly reported inclusion and exclusion criteria

Articles that meet PRISMA definition of a systematic review

Definitions of the population(s), intervention(s), comparator(s) and outcome(s) of interest

Inclusion/exclusion criteria that are relevant in terms of the PICO framework

Reviews published in Database of Reviews of Effects (DARE)

Reviews were judged to be systematic if they synthesized peer reviewed articles

Studies meeting minimum methodological standards

Reference to study designs

Data extraction

Systematic data collection

Systematic methods to extract data

Explicit methods to collect data

Data extraction by 2 independent reviewers

Reported data abstraction from trials

Independent data extraction

Explicit approach to extracting

Organized approach to extracting

Explicit methods to extract data

Performed data extraction

Extracting the information from the studies following a priori protocol

Quality, bias, appraisal, validity

Quality assessment of evidence

Critical appraisal of the studies

Risk of bias assessment

Systematic methods to critically appraise relevant research

Explicit methods to critically appraise relevant research

Reported validity assessment

Attempt to appraise all empirical evidence

Full assessment of methodological quality of included studies

Consideration of internal and external validity of the research

Provided sufficient details about individual included studies to enable assessment of quality by a reader

Reported at least one or more aspects of validity assessment of original studies

Transparent procedures to evaluate relevant research

Full report of methodological quality of included studies

Transparent process to minimize risk of bias

Explicit approach to critically evaluating studies

Organized approach to critically evaluating empirical literature

Systematic approach for assessing the studies

Reproducible approach for assessing the studies

Assessed methodological features of the included studies

Adequate methods to appraise included studies

Transparent methodological criteria are used to exclude papers that do not meet an explicit methodological benchmark

Evaluate the retrieved studies using prospectively defined methodological criteria

Analysis, synthesis

Synthesis of results

Presence of meta-analysis

Systematic methods of analysis of studies

Explicit methods to analyze data

Systematic synthesis of findings

Quantitative synthesis

Synthesis of the included evidence, whether narrative or quantitative

Attempt to synthesize all empirical evidence

Systematic analysis of results

Unbiased synthesis of study findings

Transparent procedures to synthesize the results of relevant research

Analyze results appropriately

Systematic analysis

Plausible analysis of data

Plausible synthesis of data

Summary of results

Systematic analysis

Meta-analysis or best evidence synthesis

Formal analysis contained in the methods

Makes judgement about research question

Relying on statistical significance to make judgments about what works

Transparent process of interpretation of the findings of the studies included in the review

Rigorous conclusions about outcomes

Describing included studies

Systematic presentation of characteristics of included studies

Systematic synthesis of characteristics of included studies

Clearly identified all included studies

Reported trial characteristics

Systematic presentation of main information

Described main characteristics of included studies

Adequate methods to describe included studies

Description of the number and nature of included studies

Description of the types of primary studies included

Accounted for identified studies

Reporting

Used PRISMA or predecessor guidelines for reporting

Presented results appropriately

Systematic presentation of findings

Flow chart present

Reported level of evidence for their recommendations

Reported sufficient information to allow a level of evidence grading

Published in a journal conforming to PRISMA standards

A review that has methods and results section

Unclear

“It was apparent in the text that a systematic review had been undertaken”

“Reviews were included if they were systematic”