Table 2 Questions for assessing external validity (EV)
Categories | Items | + | (+) | - | c.b.e. |
|---|---|---|---|---|---|
Study population – assessment of selection bias (related to EV) | • To what extent do the inclusion and exclusion criteria (where relevant, other selection criteria) define the "everyday or target population" of the intervention? | ☐ | ☐ | ☐ | ☐ |
• Does the applied diagnostic procedure reflect everyday conditions and the everyday possibilities (access, necessity) respectively? | ☐ | ☐ | ☐ | ☐ | |
• Are the diagnostic procedures and evaluations performed by persons with similar qualification and experience as in everyday practice? | ☐ | ☐ | ☐ | ☐ | |
• Does the study population reflect the everyday population in terms of: | ☐ | ☐ | ☐ | ☐ | |
○ Severity of the illness | ☐ | ☐ | ☐ | ☐ | |
○ Duration of illness | ☐ | ☐ | ☐ | ☐ | |
○ Intra-individual variability | ☐ | ☐ | ☐ | ☐ | |
○ Age | ☐ | ☐ | ☐ | ☐ | |
○ Gender | ☐ | ☐ | ☐ | ☐ | |
○ Further socio-demographic characteristics | ☐ | ☐ | ☐ | ☐ | |
○ Therapy preferences and expectations | ☐ | ☐ | ☐ | ☐ | |
○ Symptoms of side effects of the interventions | ☐ | ☐ | ☐ | ☐ | |
○ Accompanying illnesses | ☐ | ☐ | ☐ | ☐ | |
○ Accompanying medication | ☐ | ☐ | ☐ | ☐ | |
○ Further prognostic or therapy relevant parameters? | ☐ | ☐ | ☐ | ☐ | |
• Has the structural similarity between the study and the everyday population or target population been tested? | ☐ | ☐ | ☐ | ☐ | |
Intervention und control – assessment of performance bias (related to EV) | • Does the preparation (medication, other medicinal products, other kind of interventions) reflect the usual treatment? | ☐ | ☐ | ☐ | ☐ |
• In case of medication, does the dosage reflect the usual treatment? (Is dose modification possible?) | ☐ | ☐ | ☐ | ☐ | |
• Does the type of administration reflect the usual treatment? | ☐ | ☐ | ☐ | ☐ | |
• Does the intervention duration reflect the usual treatment duration? | ☐ | ☐ | ☐ | ☐ | |
• Are the permitted accompanying treatments the usual accompanying treatments? | ☐ | ☐ | ☐ | ☐ | |
• Does the study situation reflect the common treatment situation? | ☐ | ☐ | ☐ | ☐ | |
• Are the interventions carried out by therapists with similar qualifications and experience as in everyday practice? | ☐ | ☐ | ☐ | ☐ | |
Outcome measurements, results and evaluation – assessment of detection and attrition bias (related to EV) | • Are the chosen outcomes practice and patient relevant? (E.g. no surrogate parameter, are individual therapy goals defined?) | ☐ | ☐ | ☐ | ☐ |
• Were the following important outcomes considered: quality of life, subjective health, patient's general evaluations, compliance, reasons for dropout, use of accompanying treatments, rebound effect following termination of treatment (or, for example, symptom deceit)? | ☐ | ☐ | ☐ | ☐ | |
• Are the test procedures used in usual practice? | ☐ | ☐ | ☐ | ☐ | |
• Are the tests and evaluations performed by persons with similar qualifications and experience as in every day practice? | ☐ | ☐ | ☐ | ☐ | |
• Are the differences clinically relevant? | ☐ | ☐ | ☐ | ☐ | |
• Were sufficient data collected to cover the intra-individual variability? | ☐ | ☐ | ☐ | ☐ | |
• Do the test conditions reflect the everyday practice? | ☐ | ☐ | ☐ | ☐ | |
• Does the dropout rate reflect everyday experience? Are the reasons for dropout registered (e.g. adverse effects, insufficient effect), so that the significance for the everyday effectiveness can be assessed? | ☐ | ☐ | ☐ | ☐ | |
• Is clinical relevance considered in the conclusion? | ☐ | ☐ | ☐ | ☐ | |
Study design and Setting (related to EV) | • Is the research question clinically relevant? | ☐ | ☐ | ☐ | ☐ |
• Does the study design ensure a high EV? | ☐ | ☐ | ☐ | ☐ | |
• Does the study setting reflect the everyday conditions? | ☐ | ☐ | ☐ | ☐ | |
• Are the investigators the regular contact persons (e.g. GP or relevant clinic doctor, or are they at least comparable in terms of training, status, experience, preferences; does the number of contact people reflect the usual setting)? | ☐ | ☐ | ☐ | ☐ | |
• Does the doctor/therapist-patient relationship reflect the everyday conditions (e.g. frequency of contact, constant contact person)? | ☐ | ☐ | ☐ | ☐ |