Table 1 Definitions of key terms related to quality appraisal

Critical Appraisal (CA)

Critical appraisal is the process of carefully and systematically examining research to judge its trustworthiness, value and relevance in a particular context (Burls, [34]).

Study quality

Study quality is the extent to which a study is conducted to the highest methodological standards possible (Büttner et al., [29, 30]). Study quality is a multidimensional term referring to a set of parameters in the design, conduct and reporting of a study that reflects the validity of the outcome, related to the external (relevance and applicability) and internal validity and the statistical model used (Verhagen et al., [35]). Therefore, by assessing the study quality (Quality Assessment), one should be able to make informed judgements on a study’s trustworthiness and its value and relevance in a particular context.

Reporting quality

Reporting quality refers to the extent to which a set of parameters in the design and conduct of a study have been described to allow judgements on relevance and RoB. The purpose of reporting quality is to provide complete and transparent information about a study’s design, conduct, analysis, and results (Büttner et al., [29, 30]).

Methodological quality

Methodological quality or methodological review refers to the extent to which the study has been executed, for example, whether randomization or blinding (of participants and investigators) were done and how they were done. Notably, a randomized controlled trial (RCT) that cannot blind participants might be considered high-quality because it may be the only way for investigators to conduct such a RCT (Büttner et al., [29, 30]).

Risk of bias (RoB)

The term RoB is often used interchangeably with methodological quality or review, although the two terms are different. Bias is a systematic error, or deviation from the true findings, in results or inferences (which should not be confused with imprecision-a random error). Risk of bias refers to the likelihood that features of the study design or conduct will give misleading results or inferences. Notably, not all methodological shortcomings (low methodological quality) may result in biased estimates (or a high risk of bias)

Quality checklist

Quality checklists contain items that relate to study quality without assigning numeric values or producing a summary score (Büttner et al., [29, 30]).

Quality scale

Quality scales assign numeric values to scale items and combine information about several methodological features in a study to produce a summary score (Büttner et al., [29, 30]).

Domain-based RoB tools

Domain-based tools evaluate study limitations in specific domains that represent different biases. Example include bias arising from the randomization process or selection of participants into the study (Büttner et al., [29, 30]).

Standardized tool

A standardized tool is an instrument that is evidence-based, scientifically developed and tested for its psychometric properties (reliability, reproducibility, validity and feasibility). Therefore a standardized tool offers consistent procedures and uniform application, and it has the potential to compare findings across studies.

Technical document

A document containing information created to describe (in technical language) how the empirical elicitation of HSUVs or how the QA of HSUVs should be conducted. This can be in the form of recommendations or guidelines.