CHAMP: Efficient Annotation and Consolidation of Cluster Hierarchies
This tool addresses the need for faster and more reliable annotation of complex hierarchies in NLP, though it is incremental as it builds on existing annotation approaches.
The authors tackled the problem of efficiently annotating hierarchical cluster structures in NLP tasks by introducing CHAMP, an open-source tool that reduces annotation time compared to pairwise methods while ensuring transitivity.
Various NLP tasks require a complex hierarchical structure over nodes, where each node is a cluster of items. Examples include generating entailment graphs, hierarchical cross-document coreference resolution, annotating event and subevent relations, etc. To enable efficient annotation of such hierarchical structures, we release CHAMP, an open source tool allowing to incrementally construct both clusters and hierarchy simultaneously over any type of texts. This incremental approach significantly reduces annotation time compared to the common pairwise annotation approach and also guarantees maintaining transitivity at the cluster and hierarchy levels. Furthermore, CHAMP includes a consolidation mode, where an adjudicator can easily compare multiple cluster hierarchy annotations and resolve disagreements.