Skip to main content

Table 2 Outcome variables assessed

From: Crowdsourcing citation-screening in a mixed-studies systematic review: a feasibility study

Outcome variable

Definition

Final sensitivity

The number of citations deemed relevant by the research team (included in the final set of studies for the review after both screening and full-text review) that were correctly identified by the crowd (true positives), divided by the number of true positives plus the number of citations included in the final set of studies by the research team that were not included by the crowd (false negative).

Screening specificity

The number of citations excluded by the crowd that were also excluded from the final set of studies by the research team (true negative), divided by the number of true negatives plus the number of citations included by the crowd that were not deemed relevant by the research team after both screening and full-text review (false positive).

Efficiency

Total time taken for the crowd versus the research team to complete the screening task.