Skip to main content

Table 2 Questions for assessing external validity (EV)

From: Checklist for the qualitative evaluation of clinical studies with particular focus on external validity and model validity

Categories

Items

+

(+)

-

c.b.e.

Study population – assessment of selection bias (related to EV)

• To what extent do the inclusion and exclusion criteria (where relevant, other selection criteria) define the "everyday or target population" of the intervention?

 

• Does the applied diagnostic procedure reflect everyday conditions and the everyday possibilities (access, necessity) respectively?

 

• Are the diagnostic procedures and evaluations performed by persons with similar qualification and experience as in everyday practice?

 

• Does the study population reflect the everyday population in terms of:

 

    Severity of the illness

 

    Duration of illness

 

    Intra-individual variability

 

    Age

 

    Gender

 

    Further socio-demographic characteristics

 

    Therapy preferences and expectations

 

    Symptoms of side effects of the interventions

 

    Accompanying illnesses

 

    Accompanying medication

 

    Further prognostic or therapy relevant parameters?

 

• Has the structural similarity between the study and the everyday population or target population been tested?

Intervention und control – assessment of performance bias (related to EV)

• Does the preparation (medication, other medicinal products, other kind of interventions) reflect the usual treatment?

 

• In case of medication, does the dosage reflect the usual treatment? (Is dose modification possible?)

 

• Does the type of administration reflect the usual treatment?

 

• Does the intervention duration reflect the usual treatment duration?

 

• Are the permitted accompanying treatments the usual accompanying treatments?

 

• Does the study situation reflect the common treatment situation?

 

• Are the interventions carried out by therapists with similar qualifications and experience as in everyday practice?

Outcome measurements, results and evaluation – assessment of detection and attrition bias (related to EV)

• Are the chosen outcomes practice and patient relevant? (E.g. no surrogate parameter, are individual therapy goals defined?)

 

• Were the following important outcomes considered: quality of life, subjective health, patient's general evaluations, compliance, reasons for dropout, use of accompanying treatments, rebound effect following termination of treatment (or, for example, symptom deceit)?

 

• Are the test procedures used in usual practice?

 

• Are the tests and evaluations performed by persons with similar qualifications and experience as in every day practice?

 

• Are the differences clinically relevant?

 

• Were sufficient data collected to cover the intra-individual variability?

 

• Do the test conditions reflect the everyday practice?

 

• Does the dropout rate reflect everyday experience? Are the reasons for dropout registered (e.g. adverse effects, insufficient effect), so that the significance for the everyday effectiveness can be assessed?

 

• Is clinical relevance considered in the conclusion?

Study design and Setting (related to EV)

• Is the research question clinically relevant?

 

• Does the study design ensure a high EV?

 

• Does the study setting reflect the everyday conditions?

 

• Are the investigators the regular contact persons (e.g. GP or relevant clinic doctor, or are they at least comparable in terms of training, status, experience, preferences; does the number of contact people reflect the usual setting)?

 

• Does the doctor/therapist-patient relationship reflect the everyday conditions (e.g. frequency of contact, constant contact person)?

  1. + Matches completely/is completely fulfilled
  2. (+) Matches incompletely but sufficiently/is only partly but sufficiently fulfilled
  3. - Does not match or matches insufficiently/is insufficiently fulfilled
  4. c.b.e. Can not be evaluated