Skip to main content

Table 3 The modified version of the psychometric quality rating set out by (Terwee et al., 2007) and (Cordier et al., [30])

From: Comparison of psychometric properties between recall methods of interview-based physical activity questionnaires: a systematic review

Psychometric property Score a Quality Criteria b
Content validity + A clear description is provided of the measurement aim, the target population, the concepts that are being measured, and the item selection AND target population and (investigators OR experts) were involved in item selection
? A clear description of above-mentioned aspects is lacking OR only target population involved OR doubtful design or method
No target population involvement
± Conflicting results
NR No information found on target population involvement
NE Not evaluated
Structural validityc + Factors should explain at least 50% of the variance
? Explained variance not mentioned
Factors explain < 50% of the variance
± Conflicting results
NR No information found on structural validity
NE Not evaluated
Hypothesis testingc + Specific hypotheses were formulated AND at least 75% of the results are in accordance with these hypotheses; Convergent validity: correlation
between similar assessments is at a statistically significant level (p < 0.05) and strength of relationship is ≥0.5 which is consistent with the hypothesis; Discriminant validity: uses appropriate statistical analysis (e.g., t-test p < 0.05 or Cohen’s d effect size ≥0.5)
? Doubtful design or method (e.g., no hypotheses)
Less than 75% of hypotheses were confirmed, despite adequate design and methods; Convergent validity: correlation between similar assessments is not at a statistically significant level (p ≥ 0.05) and strength of relationship is < 0.5 which is inconsistent with hypothesis
± Conflicting results between studies within the same manual
NR No information found on hypotheses testing
NE Not evaluated
Internal consistency + Factor analyses performed on adequate sample size (7 * # items and 100) AND Cronbach’s alpha(s) calculated per dimension AND Cronbach’s alpha(s) between 0.70 and 0.95
? No factor analysis OR doubtful design or method
Cronbach’s alpha(s) < 0.70 or > 0.95, despite adequate design and method
± Conflicting results
NR No information found on internal consistency
NE Not evaluated
Reliability + ICC or weighted Kappa 0.70
? Doubtful design or method (e.g., time interval not mentioned)
ICC or weighted Kappa < 0.70, despite adequate design and method
± Conflicting results
NR No information found on reliability
NE Not evaluated
Measurement errord + MIC < SDC OR MIC outside the LOA OR convincing arguments that agreement is acceptable
? Doubtful design or method OR (MIC not defined AND no convincing arguments that agreement is acceptable)
MIC SDC OR MIC equals or inside LOA, despite adequate design and method
± Conflicting results
NR No information found on measurement error
NE Not evaluated
  1. Notes. aScores: + = positive rating,? = indeterminate rating, — = negative rating, ± = conflicting data, NR = not reported, NE = not evaluated (for study of poor methodological quality according to COSMIN rating, data are excluded from further evaluation
  2. bDoubtful design or method is assigned when a clear description of the design or methods of the study is lacking, sample size smaller than 50 subjects (should be at least 50 in every subgroup analysis), or any important methodological weakness in the design or execution of the study
  3. cHypothesis testing: all correlations should be statistically significant (if not, these hypotheses are not confirmed) AND these correlations should be at least moderate (r > 0.5)
  4. dMeasurement error: MIC minimal important change, SDC smallest detectable change, LOA limits of agreement