From: Crowdsourcing citation-screening in a mixed-studies systematic review: a feasibility study
Outcome variable | Definition |
---|---|
Final sensitivity | The number of citations deemed relevant by the research team (included in the final set of studies for the review after both screening and full-text review) that were correctly identified by the crowd (true positives), divided by the number of true positives plus the number of citations included in the final set of studies by the research team that were not included by the crowd (false negative). |
Screening specificity | The number of citations excluded by the crowd that were also excluded from the final set of studies by the research team (true negative), divided by the number of true negatives plus the number of citations included by the crowd that were not deemed relevant by the research team after both screening and full-text review (false positive). |
Efficiency | Total time taken for the crowd versus the research team to complete the screening task. |