Skip to main content

Table 7 Consensus reached on standards for preferred statistical methods for measurement error (agreement)

From: COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study

Statistical methods

very good

adequate

doubtful

inadequate

7

For continuous scores: was the Standard Error of Measurement (SEM), Smallest Detectable Change (SDC), Limits of Agreement (LoA) or Coefficient of Variation (CV) calculated?

Relevance: 29/38 (76%)(R2a); wording: 22/32 (69%)(R3b); add CV: 22/30 (73%)(R3)

SEM, SDC, LoA or CV calculated; the model or formula for the SEM/SDC is described; it matches the study designc and the data

32/36 (89%)(R2)

SEM, SDC, LoA or CV calculated, but the model or formula is not described or does not optimally match the study design and evidence provided that no systematic difference has occurred

25/34 (72%)(R2)

SEMconsistency SDCconsistency or LoA or CV calculated, without knowledge about systematic difference or with evidence provided that systematic difference has occurred

SEM calculated based on Cronbach’s alpha

22/31 (71%)(R3)

OR using SD from another population

27/34 (79%)(R2)

8

For dichotomous/ nominal/ ordinal scores: Was the percentage specific (e.g. positive and negative) agreement calculated?

24/35 (69%)(R2)

% specific agreement calculated

% agreement calculated

  
  1. a R2: consensus reached in round 2; b R3: consensus reached in round 3; c Based on panelists’ suggestions the steering committee decided after round 3 to use the word ‘study design’ instead of ‘reviewer constructed research question’