Table 1 Results of studies quantifying the frequency of data extraction errors
Study | Studies included (reviews) | Measure | Result |
|---|---|---|---|
Carroll 2013 [16] | 8 (3) | Selection (outcome 1) | 17% (review 1 vs. reference standard); 8% (review 2 vs. reference standard) |
Selection (outcome 2) | 42% (review 1 vs. reference standard); 25% (review 2 vs. reference standard) | ||
Selection (outcome 3) | 21% (review 1 vs. reference standard); 25% (review 2 vs. reference standard) | ||
Inaccuracy (outcome 1) | 8% (review 1 vs. reference standard); 8% (review 2 vs. reference standard) | ||
Inaccuracy (outcome 2) | 17% (review 1 vs. reference standard); 13% (review 2 vs. reference standard) | ||
Inaccuracy (outcome 3) | 13% (review 1 vs. reference standard); 8% (review 2 vs. reference standard) | ||
Difference in meta-analysis (outcome 1) | RR 1.70 (reference standard) / RR 1.71 (review 1) | ||
Difference in meta-analysis (outcome 2) | RR 0.85 (reference standard) / RR 0.87 (review 1) / RR 0.80 (review 2) | ||
Difference in meta-analysis (outcome 3) | RR 0.38 (reference standard) / RR 0.40 (review 1) | ||
Gøtzsche 2007 [8] | 54 (random selected; 27 meta-analysis) | Difference in SMD >0.1 of at least 1 of the 2 included trials | 63% |
20 (10 meta-analysisa) | Difference in SMD >0.1 of pooled effect estimate. | 70% | |
Jones 2005 [17] | NR (34) | Errors (all types) | 50% |
Correct interpretation | 23.3% | ||
Impact on results | All data-handling errors led to changes in the summary results, but none of them affected review conclusionsb | ||
Tendal 2009 [15] | 45 (10 meta-analysis) | Difference in SMD because of reviewer disagreements < 0.1 | 53% |
Difference in SMD because of reviewer disagreements < 0.1 (pooled estimates) | 31% |