Skip to main content

Table 3 Results of studies comparing different reviewer extraction methods and reviewer characteristics

From: Frequency of data extraction errors and methods to increase data extraction quality: a methodological review

 

Measure

Result (effect measure, CI or p-value)

Reviewer constellation

 

Buscemi 2006 [9]

Agreement rate

28.0% (95% CI: 25.4, 30.7, range 11.1–47.2%)

Errors (all types)

RD 21.7% (p = 0.019)

Omission

RD 6.6% (p = 0.308)

Time (min, mean)

RD −49 (p = 0.03)

Difference of pooled effect estimates *

0/0

Reviewer experience

 

Horton 2010 [14]

Errors (all types)

24.3%/26.4%/25.4% (p = 0.91)

Inaccuracy

14.3%/13.6%/15.7% (p = 0.41)

Omission

10.0%/12.1%/12.1% (p = 0.24)

Time (min, mean)

200/149/163 (p = 0.03)

MD in point estimates of meta -analysis

ns (5 outcomes)

Errors (all types)

25.0%/ 26.1%/ 24.3% (p = 0.73)

Inaccuracy

14.6%/ 13.2%/ 15.7% (p = 0.39)

Omission

10.0%/ 11.4%/ 10.7% (p = 0.53)

Time (min, mean)

198/179/ 152 (p = 0.01)

MD in point estimates of meta -analysis

ns (5 outcomes)

Errors (all types)

26.4%/27.9%/27.9% (p = 0.73)

Inaccuracy

16.4%/12.1%/15.7% (p = 0.22)

Omission

10.4%/12.1%/13.6% (p = 0.47)

Time (min, mean)

211/180/173 (p = 0.12)

MD in point estimates of meta -analysis

ns (5 outcomes)

Tendal 2009 [15]

Difference of SMD < 0.1

61%/46% (NR)

Difference of SMD < 0.1 (pooled estimates)

33%/27% (NR)

  1. *p < 0.05; CI confidence interval, MD mean difference, ns not statistical significant differences (according to authors, significance not specified), NR not reported, RD relative difference, SMD standardized mean difference