Skip to main content

Table 2 Questions for assessing external validity (EV)

From: Checklist for the qualitative evaluation of clinical studies with particular focus on external validity and model validity

Categories Items + (+) - c.b.e.
Study population – assessment of selection bias (related to EV) • To what extent do the inclusion and exclusion criteria (where relevant, other selection criteria) define the "everyday or target population" of the intervention? ☐ ☐ ☐ ☐
  • Does the applied diagnostic procedure reflect everyday conditions and the everyday possibilities (access, necessity) respectively? ☐ ☐ ☐ ☐
  • Are the diagnostic procedures and evaluations performed by persons with similar qualification and experience as in everyday practice? ☐ ☐ ☐ ☐
  • Does the study population reflect the everyday population in terms of: ☐ ☐ ☐ ☐
     ○ Severity of the illness ☐ ☐ ☐ ☐
     ○ Duration of illness ☐ ☐ ☐ ☐
     ○ Intra-individual variability ☐ ☐ ☐ ☐
     ○ Age ☐ ☐ ☐ ☐
     ○ Gender ☐ ☐ ☐ ☐
     ○ Further socio-demographic characteristics ☐ ☐ ☐ ☐
     ○ Therapy preferences and expectations ☐ ☐ ☐ ☐
     ○ Symptoms of side effects of the interventions ☐ ☐ ☐ ☐
     ○ Accompanying illnesses ☐ ☐ ☐ ☐
     ○ Accompanying medication ☐ ☐ ☐ ☐
     ○ Further prognostic or therapy relevant parameters? ☐ ☐ ☐ ☐
  • Has the structural similarity between the study and the everyday population or target population been tested? ☐ ☐ ☐ ☐
Intervention und control – assessment of performance bias (related to EV) • Does the preparation (medication, other medicinal products, other kind of interventions) reflect the usual treatment? ☐ ☐ ☐ ☐
  • In case of medication, does the dosage reflect the usual treatment? (Is dose modification possible?) ☐ ☐ ☐ ☐
  • Does the type of administration reflect the usual treatment? ☐ ☐ ☐ ☐
  • Does the intervention duration reflect the usual treatment duration? ☐ ☐ ☐ ☐
  • Are the permitted accompanying treatments the usual accompanying treatments? ☐ ☐ ☐ ☐
  • Does the study situation reflect the common treatment situation? ☐ ☐ ☐ ☐
  • Are the interventions carried out by therapists with similar qualifications and experience as in everyday practice? ☐ ☐ ☐ ☐
Outcome measurements, results and evaluation – assessment of detection and attrition bias (related to EV) • Are the chosen outcomes practice and patient relevant? (E.g. no surrogate parameter, are individual therapy goals defined?) ☐ ☐ ☐ ☐
  • Were the following important outcomes considered: quality of life, subjective health, patient's general evaluations, compliance, reasons for dropout, use of accompanying treatments, rebound effect following termination of treatment (or, for example, symptom deceit)? ☐ ☐ ☐ ☐
  • Are the test procedures used in usual practice? ☐ ☐ ☐ ☐
  • Are the tests and evaluations performed by persons with similar qualifications and experience as in every day practice? ☐ ☐ ☐ ☐
  • Are the differences clinically relevant? ☐ ☐ ☐ ☐
  • Were sufficient data collected to cover the intra-individual variability? ☐ ☐ ☐ ☐
  • Do the test conditions reflect the everyday practice? ☐ ☐ ☐ ☐
  • Does the dropout rate reflect everyday experience? Are the reasons for dropout registered (e.g. adverse effects, insufficient effect), so that the significance for the everyday effectiveness can be assessed? ☐ ☐ ☐ ☐
  • Is clinical relevance considered in the conclusion? ☐ ☐ ☐ ☐
Study design and Setting (related to EV) • Is the research question clinically relevant? ☐ ☐ ☐ ☐
  • Does the study design ensure a high EV? ☐ ☐ ☐ ☐
  • Does the study setting reflect the everyday conditions? ☐ ☐ ☐ ☐
  • Are the investigators the regular contact persons (e.g. GP or relevant clinic doctor, or are they at least comparable in terms of training, status, experience, preferences; does the number of contact people reflect the usual setting)? ☐ ☐ ☐ ☐
  • Does the doctor/therapist-patient relationship reflect the everyday conditions (e.g. frequency of contact, constant contact person)? ☐ ☐ ☐ ☐
  1. + Matches completely/is completely fulfilled
  2. (+) Matches incompletely but sufficiently/is only partly but sufficiently fulfilled
  3. - Does not match or matches insufficiently/is insufficiently fulfilled
  4. c.b.e. Can not be evaluated
Back to article page

BMC Medical Research Methodology

ISSN: 1471-2288

Contact us