Skip to main content

Advertisement

Table 3 Results of studies comparing different reviewer extraction methods and reviewer characteristics

From: Frequency of data extraction errors and methods to increase data extraction quality: a methodological review

  Measure Result (effect measure, CI or p-value)
Reviewer constellation  
Buscemi 2006 [9] Agreement rate 28.0% (95% CI: 25.4, 30.7, range 11.1–47.2%)
Errors (all types) RD 21.7% (p = 0.019)
Omission RD 6.6% (p = 0.308)
Time (min, mean) RD −49 (p = 0.03)
Difference of pooled effect estimates * 0/0
Reviewer experience  
Horton 2010 [14] Errors (all types) 24.3%/26.4%/25.4% (p = 0.91)
Inaccuracy 14.3%/13.6%/15.7% (p = 0.41)
Omission 10.0%/12.1%/12.1% (p = 0.24)
Time (min, mean) 200/149/163 (p = 0.03)
MD in point estimates of meta -analysis ns (5 outcomes)
Errors (all types) 25.0%/ 26.1%/ 24.3% (p = 0.73)
Inaccuracy 14.6%/ 13.2%/ 15.7% (p = 0.39)
Omission 10.0%/ 11.4%/ 10.7% (p = 0.53)
Time (min, mean) 198/179/ 152 (p = 0.01)
MD in point estimates of meta -analysis ns (5 outcomes)
Errors (all types) 26.4%/27.9%/27.9% (p = 0.73)
Inaccuracy 16.4%/12.1%/15.7% (p = 0.22)
Omission 10.4%/12.1%/13.6% (p = 0.47)
Time (min, mean) 211/180/173 (p = 0.12)
MD in point estimates of meta -analysis ns (5 outcomes)
Tendal 2009 [15] Difference of SMD < 0.1 61%/46% (NR)
Difference of SMD < 0.1 (pooled estimates) 33%/27% (NR)
  1. *p < 0.05; CI confidence interval, MD mean difference, ns not statistical significant differences (according to authors, significance not specified), NR not reported, RD relative difference, SMD standardized mean difference