Skip to main content

Table 3 Questions for assessing model validity (MV)

From: Checklist for the qualitative evaluation of clinical studies with particular focus on external validity and model validity

Categories Items + (+) - c.b.e.
Study population – assessment of selection bias (related to MV) • To what extent do the inclusion and exclusion criteria and, where relevant, other selection criteria define an optimal population with respect to the test intervention? (An optimal population will show the highest benefit from the applied intervention).
  • Is the applied diagnosis and/or classification relevant for the intervention?
  • Are relevant subgroups considered?
  • Does the diagnostic procedure optimally reflect the aptitude for the intervention?
  • Are the diagnostic procedures performed by qualified and experienced physicians?
  • Does the study population reflect the ideal population in terms of:
      Severity of the illness
      Duration of the illness
      Intra-individual variability
      Age
      Gender
      Further socio-demographic characteristics
      Therapy preferences and expectations
      Symptoms of the side effects of the interventions
      Accompanying illnesses
      Accompanying medication
      Further prognostic or therapy relevant parameters? (The above listed factors can influence the measurement of outcomes so that floor/ceiling-effects may occur)
  • (Is the structural similarity between the study population and the ideal population for the intervention tested? – A fairly hypothetical question)
Intervention und control – assessment of performance bias (related to MV) • Is the investigational intervention the optimal treatment?
  • In case of medication, is the dosage the optimal treatment?
  • Is the application the optimal treatment?
  • Is the intervention duration the optimal treatment duration? (Are there signs of marketing strategies of pharmaceutical companies?)
  • Are the permitted accompanying treatments the optimal accompanying treatments?
  • Are the study conditions the optimal conditions for the intervention?
  • Are the interventions carried out by qualified and experienced therapists?
Outcome measurements, results and evaluation – assessment of detection and attrition bias (related to MV) • Do the outcome parameters reflect the effects of the intervention in the best possible manner? (Also consider here floor/ceiling effects).
  • Do the applied test procedures best reflect the chosen outcomes of intervention effects?
  • Are the test conditions appropriate to optimally evaluate the intervention efficacy?
  • Is the length of follow-up sufficient to detect the intervention effects (including adverse effects and rebound effects following termination of the treatment)?
  • Is there an analysis carried out that considers the actually applied treatment interventions (PP analysis)?
  • Are tests and evaluations carried out by qualified and experienced examiners?
  • (Retrospectively, were optimal conditions given for the identification of the intervention efficacy?)
Study design and setting (related to MV) • Does the research question reflect the optimal conditions for the intervention?
  • Does the study design ensure a high level of MV?
  • Does the study setting reflect the optimal treatment conditions?
  • Do the therapists/investigators have adequate experience with the intervention or the indication in question?
  • Do therapists/investigators and patient have a positive attitude towards the intervention?
  1. + Matches completely/is completely fulfilled
  2. (+) Matches incompletely but sufficiently/is only partly but sufficiently fulfilled
  3. - Does not match or matches insufficiently/is insufficiently fulfilled
  4. c.b.e. Can not be evaluated