Skip to main content

Table 2 Questions for assessing external validity (EV)

From: Checklist for the qualitative evaluation of clinical studies with particular focus on external validity and model validity

Categories Items + (+) - c.b.e.
Study population – assessment of selection bias (related to EV) • To what extent do the inclusion and exclusion criteria (where relevant, other selection criteria) define the "everyday or target population" of the intervention?
  • Does the applied diagnostic procedure reflect everyday conditions and the everyday possibilities (access, necessity) respectively?
  • Are the diagnostic procedures and evaluations performed by persons with similar qualification and experience as in everyday practice?
  • Does the study population reflect the everyday population in terms of:
      Severity of the illness
      Duration of illness
      Intra-individual variability
      Further socio-demographic characteristics
      Therapy preferences and expectations
      Symptoms of side effects of the interventions
      Accompanying illnesses
      Accompanying medication
      Further prognostic or therapy relevant parameters?
  • Has the structural similarity between the study and the everyday population or target population been tested?
Intervention und control – assessment of performance bias (related to EV) • Does the preparation (medication, other medicinal products, other kind of interventions) reflect the usual treatment?
  • In case of medication, does the dosage reflect the usual treatment? (Is dose modification possible?)
  • Does the type of administration reflect the usual treatment?
  • Does the intervention duration reflect the usual treatment duration?
  • Are the permitted accompanying treatments the usual accompanying treatments?
  • Does the study situation reflect the common treatment situation?
  • Are the interventions carried out by therapists with similar qualifications and experience as in everyday practice?
Outcome measurements, results and evaluation – assessment of detection and attrition bias (related to EV) • Are the chosen outcomes practice and patient relevant? (E.g. no surrogate parameter, are individual therapy goals defined?)
  • Were the following important outcomes considered: quality of life, subjective health, patient's general evaluations, compliance, reasons for dropout, use of accompanying treatments, rebound effect following termination of treatment (or, for example, symptom deceit)?
  • Are the test procedures used in usual practice?
  • Are the tests and evaluations performed by persons with similar qualifications and experience as in every day practice?
  • Are the differences clinically relevant?
  • Were sufficient data collected to cover the intra-individual variability?
  • Do the test conditions reflect the everyday practice?
  • Does the dropout rate reflect everyday experience? Are the reasons for dropout registered (e.g. adverse effects, insufficient effect), so that the significance for the everyday effectiveness can be assessed?
  • Is clinical relevance considered in the conclusion?
Study design and Setting (related to EV) • Is the research question clinically relevant?
  • Does the study design ensure a high EV?
  • Does the study setting reflect the everyday conditions?
  • Are the investigators the regular contact persons (e.g. GP or relevant clinic doctor, or are they at least comparable in terms of training, status, experience, preferences; does the number of contact people reflect the usual setting)?
  • Does the doctor/therapist-patient relationship reflect the everyday conditions (e.g. frequency of contact, constant contact person)?
  1. + Matches completely/is completely fulfilled
  2. (+) Matches incompletely but sufficiently/is only partly but sufficiently fulfilled
  3. - Does not match or matches insufficiently/is insufficiently fulfilled
  4. c.b.e. Can not be evaluated