Model-based annotation of coreference
This addresses the problem of inefficient and inconsistent coreference annotation for NLP researchers, though it is incremental as it builds on existing knowledge base linking methods.
The paper tackles the unnatural task of coreference annotation by introducing model-based annotation, where pronouns are assigned to entities linked to a knowledge base, resulting in faster annotation and higher inter-annotator agreement.
Humans do not make inferences over texts, but over models of what texts are about. When annotators are asked to annotate coreferent spans of text, it is therefore a somewhat unnatural task. This paper presents an alternative in which we preprocess documents, linking entities to a knowledge base, and turn the coreference annotation task -- in our case limited to pronouns -- into an annotation task where annotators are asked to assign pronouns to entities. Model-based annotation is shown to lead to faster annotation and higher inter-annotator agreement, and we argue that it also opens up for an alternative approach to coreference resolution. We present two new coreference benchmark datasets, for English Wikipedia and English teacher-student dialogues, and evaluate state-of-the-art coreference resolvers on them.