Table 1 Methodological key items for systematic reviews or meta-analyses of observational studies

Protocol development

A protocol is written in the preliminary stages of a research synthesis to describe the rational of the review and the methods that will be used to minimise the potential for bias in the review process.

Research question

The research question is defined a priori as for any research project. It sets the scope of the review and guides subsequent decisions about the methods to be used to answer the particular research question.

Search strategy

The search strategy refers to the methods employed to conduct a methodologically sound search and might include information as the data sources used and the specific terms applied in distinct databases. The search locates articles relevant to answer the a priori defined research question.

Study eligibility

Study eligibility is assessed according to pre-defined eligibility criteria related to the study itself such as the study design, the study population, as well as the exposure/s and outcome/s of interest but also to aspects such as the language and year of publication. Usually two reviewers assess each study for eligibility to reduce errors and bias. Specifying which features should be covered by eligibility criteria might be more difficult for observational studies than for RCTs as observational studies cover a broader range of research questions and have more variability in design.

Data extraction

Data extraction is performed according to a standardised form that has been finalised during pilot extraction. Usually two reviewers extract data for each study for eligibility to reduce errors and bias. Data extraction for observational studies might be less straight forward than for RCTs because multiple analyses may have been conducted (e.g. unadjusted and adjusted, with analyses adjusting for different sets of potential confounders), and each observational study design will have different data to be extracted.

Considering different study designs

Before starting evidence synthesis of observational studies, reviewers must consider which study designs to include as well as how to approach the analysis of data from different study designs. This adds complexity over evidence synthesis that considers RCTs only.

Risk of bias assessment

A risk of bias assessment of all primary studies included is important for all systematic reviews and meta-analyses. This assessment allows a better understanding of how bias may have affect results of studies, and subsequently the results of evidence synthesis. Risk of bias assessment of observational studies may be more complex than in RCTs since observational studies are likely to be prone to bias and confounders.

Publication bias

Publication bias needs to be considered in any systematic review and meta-analysis as only about half of all completed research projects reach publication in an indexed journal.

Heterogeneity

The term heterogeneity refers to differences in results between studies. When heterogeneity exists between studies, it is important to understand why as this will alter the conclusions drawn by the review. An exploration of heterogeneity might be particularly important when reviewing observational studies given the range of study designs and the potential risk of bias in observational studies.

Statistical analysis

Statistical analysis in the context of meta-analysis refers to the mathematical analysis and combination of the results of the included primary studies. Important aspects to consider are whether to pool data to provide a single effect in light of observed heterogeneity and how to choose the statistical model to be employed (e.g. fixed or random-effects model). These decisions might need more careful consideration when reviewing observational studies given the range of study designs and the potential risk of bias in observational studies.