CrowdRev: A platform for Crowd-based Screening of Literature Reviews
This addresses the time and cost inefficiencies in literature review screening for researchers, though it appears incremental as it builds on existing crowd-based methods.
The authors tackled the problem of screening literature reviews by developing CrowdRev, a crowd and crowd+AI system that achieves the same quality as author classification at a fraction of the cost and near-instantly.
In this paper and demo we present a crowd and crowd+AI based system, called CrowdRev, supporting the screening phase of literature reviews and achieving the same quality as author classification at a fraction of the cost, and near-instantly. CrowdRev makes it easy for authors to leverage the crowd, and ensures that no money is wasted even in the face of difficult papers or criteria: if the system detects that the task is too hard for the crowd, it just gives up trying (for that paper, or for that criteria, or altogether), without wasting money and never compromising on quality.