A Neural Span-Based Continual Named Entity Recognition Model
This work addresses the incremental learning challenge for NER in applications like personal assistants, though it is incremental as it adapts existing span-based methods to continual learning.
The paper tackles the problem of continual learning for named entity recognition (NER) where entity types increase over time, proposing SpanKL, a span-based model with knowledge distillation and multi-label prediction, which significantly outperforms previous state-of-the-art methods on synthetic datasets from OntoNotes and Few-NERD, achieving the smallest gap from the upper bound.
Named Entity Recognition (NER) models capable of Continual Learning (CL) are realistically valuable in areas where entity types continuously increase (e.g., personal assistants). Meanwhile the learning paradigm of NER advances to new patterns such as the span-based methods. However, its potential to CL has not been fully explored. In this paper, we propose SpanKL, a simple yet effective Span-based model with Knowledge distillation (KD) to preserve memories and multi-Label prediction to prevent conflicts in CL-NER. Unlike prior sequence labeling approaches, the inherently independent modeling in span and entity level with the designed coherent optimization on SpanKL promotes its learning at each incremental step and mitigates the forgetting. Experiments on synthetic CL datasets derived from OntoNotes and Few-NERD show that SpanKL significantly outperforms previous SoTA in many aspects, and obtains the smallest gap from CL to the upper bound revealing its high practiced value. The code is available at https://github.com/Qznan/SpanKL.