RuREBus: a Case Study of Joint Named Entity Recognition and Relation Extraction from e-Government Domain
This highlights limitations in current NER and RE technologies for domain-specific, non-English applications, indicating an incremental study that points to the need for more sophisticated methods.
The paper tackled the problem of applying named entity recognition and relation extraction to a novel e-government corpus in a non-English language with a unique annotation scheme, finding that state-of-the-art transformer models showed modest performance and fine-tuning on unlabeled data did not significantly improve results.
We show-case an application of information extraction methods, such as named entity recognition (NER) and relation extraction (RE) to a novel corpus, consisting of documents, issued by a state agency. The main challenges of this corpus are: 1) the annotation scheme differs greatly from the one used for the general domain corpora, and 2) the documents are written in a language other than English. Unlike expectations, the state-of-the-art transformer-based models show modest performance for both tasks, either when approached sequentially, or in an end-to-end fashion. Our experiments have demonstrated that fine-tuning on a large unlabeled corpora does not automatically yield significant improvement and thus we may conclude that more sophisticated strategies of leveraging unlabelled texts are demanded. In this paper, we describe the whole developed pipeline, starting from text annotation, baseline development, and designing a shared task in hopes of improving the baseline. Eventually, we realize that the current NER and RE technologies are far from being mature and do not overcome so far challenges like ours.