Skip to main content

Table 4 Information to be collected for ascertaining reference values for external validity (EV) and model validity (MV)

From: Checklist for the qualitative evaluation of clinical studies with particular focus on external validity and model validity

Categories Items
Study population • Characterisation of the routine and optimal indication and population, respectively for the therapy (where appropriate with modifications); for what kind of patients with the same or similar conditions are alternative therapies favoured?
  • Which diagnostic tests are routinely used in the diagnostic procedure?
  • Where possible, typical and optimal patient characterisation in relation to:
      Severity of the illness
      Duration of illness
      Intra-individual variability
      Further socio-demographic characteristics
      Therapy preferences and expectations
      Symptoms of adverse effects of the interventions
      Accompanying illnesses
      Accompanying medication
      Further prognostic or therapy relevant parameters?
  • Therapy expectation for the "everyday" or "optimal" population (are these discussed with the patients)? Data on course of illness without treatment?
  • Other than the aforementioned characteristics?
  • Which patients are not suitable for the intervention and why?
Intervention und Control • Which are the most commonly prescribed or best effective drugs/interventions for the indication/illness in question (gold-standard), and how do they differ from the investigational drug/intervention? When will the study intervention be selected, when the therapy alternative?
  • In case of medication, usual dosage of the therapy; is this related to the optimal effect? (Or is this a reduced dose due to adverse effects?)
  • Routine method of application of the therapy, is this also the optimal application?
  • Routine therapy duration? (Or by continuous application: after which period of time is a treatment-free interval considered? When are possible changes to alternative medications considered?)
  • Usual accompanying treatments in the therapy of the illness?
  • Usual accompanying therapies for commonly associated illnesses? Are interactions with investigational or similar drug/intervention known?
  • Are typical characteristics of the drug/intervention known (e.g. taste, odour, or local irritation following application, whereby the intervention can become "un-blinded")?
  • Known adverse effects of the intervention?
  • Context factors of the usual or optimal interventions treatment (e.g. individual modification of therapy, therapy expectation of the physician and patient, kind of medical care, accessibility to the therapy)?
Outcome measurements, results and evaluation • What are the relevant outcomes for the patients (or their relatives) in practice?
  • Which parameters are routinely assessed for the progress evaluation of the illness/indication?
  • Which outcome measurements best reflect the intervention's efficacy?
  • What are the routine assessment procedures for the chosen outcome measurements and their clinically relevant threshold values?
  • Is intra-individual variability in illness progress accounted for?
  • What are the routine conditions for the test procedure?
  • What are the optimal conditions for the representation of the intervention efficacy?
  • When will the intervention effect become apparent (earliest, latest)?
  • When are adverse or rebound effects expected after termination of the application?
  • Estimate of the compliance for the intervention (in comparison to alternative therapies), reasons for non-compliance or dropouts, for what reasons is the intervention terminated, after what average time period?
  • (Questions regarding the model validity of individual studies may at best be answered by topic experts)
Study design and setting • What are the clinically relevant research questions?
  • Characteristics of best cases/settings and worst cases/settings for the treatment?
  • Description of the standard treatment setting (and variability), and optimal setting?
  • Are there specifically relevant factors for the indication or therapy?
  • Which physicians have the most experience with the intervention or the indication (practice/clinic, speciality, level of training, experience)?
  • Typical first and subsequent contact with the intervention from the patient's perspective?
  • Time and effort for medical care in routine and optimal situations? Other relevant context factors (e.g. trust in therapy and/or physician)?
  • Time and effort for routine and optimal documentation?
  • (For hypothesis testing without comparison group: Is the reference value clinically relevant?)
  • Where appropriate: Consult specialists for an evaluation of the closeness to practice of individual studies or other factors concerning EV and MV.