Cross-lingual Named Entity Corpus for Slavic Languages
This addresses the problem of limited resources for named entity processing in Slavic languages, providing a new dataset and benchmarks for researchers in natural language processing.
The paper tackles the lack of a cross-lingual named entity corpus for Slavic languages by presenting a manually annotated corpus for six languages, achieving benchmarks with transformer models like XLM-RoBERTa-large and mT5-large.
This paper presents a corpus manually annotated with named entities for six Slavic languages - Bulgarian, Czech, Polish, Slovenian, Russian, and Ukrainian. This work is the result of a series of shared tasks, conducted in 2017-2023 as a part of the Workshops on Slavic Natural Language Processing. The corpus consists of 5 017 documents on seven topics. The documents are annotated with five classes of named entities. Each entity is described by a category, a lemma, and a unique cross-lingual identifier. We provide two train-tune dataset splits - single topic out and cross topics. For each split, we set benchmarks using a transformer-based neural network architecture with the pre-trained multilingual models - XLM-RoBERTa-large for named entity mention recognition and categorization, and mT5-large for named entity lemmatization and linking.