CoRefi: A Crowd Sourcing Suite for Coreference Annotation
This addresses the need for cheaper and more efficient coreference annotation for NLP researchers and practitioners, though it is incremental as it builds on existing crowdsourcing approaches.
The authors tackled the problem of expensive and time-consuming coreference annotation by developing CoRefi, a web-based suite for crowdsourcing, which includes guided onboarding and a novel reviewing algorithm, resulting in an open-source tool that embeds into websites and platforms.
Coreference annotation is an important, yet expensive and time consuming, task, which often involved expert annotators trained on complex decision guidelines. To enable cheaper and more efficient annotation, we present CoRefi, a web-based coreference annotation suite, oriented for crowdsourcing. Beyond the core coreference annotation tool, CoRefi provides guided onboarding for the task as well as a novel algorithm for a reviewing phase. CoRefi is open source and directly embeds into any website, including popular crowdsourcing platforms. CoRefi Demo: aka.ms/corefi Video Tour: aka.ms/corefivideo Github Repo: https://github.com/aribornstein/corefi