Skip to main content

Table 3 The modified version of the psychometric quality rating set out by (Terwee et al., 2007) and (Cordier et al., [30])

From: Comparison of psychometric properties between recall methods of interview-based physical activity questionnaires: a systematic review

Psychometric property

Score a

Quality Criteria b

Content validity

+

A clear description is provided of the measurement aim, the target population, the concepts that are being measured, and the item selection AND target population and (investigators OR experts) were involved in item selection

?

A clear description of above-mentioned aspects is lacking OR only target population involved OR doubtful design or method

–

No target population involvement

±

Conflicting results

NR

No information found on target population involvement

NE

Not evaluated

Structural validityc

+

Factors should explain at least 50% of the variance

?

Explained variance not mentioned

–

Factors explain < 50% of the variance

±

Conflicting results

NR

No information found on structural validity

NE

Not evaluated

Hypothesis testingc

+

Specific hypotheses were formulated AND at least 75% of the results are in accordance with these hypotheses; Convergent validity: correlation

between similar assessments is at a statistically significant level (p < 0.05) and strength of relationship is ≥0.5 which is consistent with the hypothesis; Discriminant validity: uses appropriate statistical analysis (e.g., t-test p < 0.05 or Cohen’s d effect size ≥0.5)

?

Doubtful design or method (e.g., no hypotheses)

–

Less than 75% of hypotheses were confirmed, despite adequate design and methods; Convergent validity: correlation between similar assessments is not at a statistically significant level (p ≥ 0.05) and strength of relationship is < 0.5 which is inconsistent with hypothesis

±

Conflicting results between studies within the same manual

NR

No information found on hypotheses testing

NE

Not evaluated

Internal consistency

+

Factor analyses performed on adequate sample size (7 * # items and 100) AND Cronbach’s alpha(s) calculated per dimension AND Cronbach’s alpha(s) between 0.70 and 0.95

?

No factor analysis OR doubtful design or method

–

Cronbach’s alpha(s) < 0.70 or > 0.95, despite adequate design and method

±

Conflicting results

NR

No information found on internal consistency

NE

Not evaluated

Reliability

+

ICC or weighted Kappa 0.70

?

Doubtful design or method (e.g., time interval not mentioned)

–

ICC or weighted Kappa < 0.70, despite adequate design and method

±

Conflicting results

NR

No information found on reliability

NE

Not evaluated

Measurement errord

+

MIC < SDC OR MIC outside the LOA OR convincing arguments that agreement is acceptable

?

Doubtful design or method OR (MIC not defined AND no convincing arguments that agreement is acceptable)

–

MIC SDC OR MIC equals or inside LOA, despite adequate design and method

±

Conflicting results

NR

No information found on measurement error

NE

Not evaluated

  1. Notes. aScores: + = positive rating,? = indeterminate rating, — = negative rating, ± = conflicting data, NR = not reported, NE = not evaluated (for study of poor methodological quality according to COSMIN rating, data are excluded from further evaluation
  2. bDoubtful design or method is assigned when a clear description of the design or methods of the study is lacking, sample size smaller than 50 subjects (should be at least 50 in every subgroup analysis), or any important methodological weakness in the design or execution of the study
  3. cHypothesis testing: all correlations should be statistically significant (if not, these hypotheses are not confirmed) AND these correlations should be at least moderate (r > 0.5)
  4. dMeasurement error: MIC minimal important change, SDC smallest detectable change, LOA limits of agreement