Regularization for Long Named Entity Recognition
This work addresses generalization issues in named entity recognition for real-world applications, particularly in biomedical and general domains, but it is incremental as it builds on existing debiasing techniques.
The paper tackles the problem of pre-trained language models being biased towards dataset patterns like entity length, which hinders generalization to unseen mentions in named entity recognition. The proposed RegLER method significantly improves predictions for long-named entities by debiasing on conjunction or special characters and alleviating class imbalance.
When performing named entity recognition (NER), entity length is variable and dependent on a specific domain or dataset. Pre-trained language models (PLMs) are used to solve NER tasks and tend to be biased toward dataset patterns such as length statistics, surface form, and skewed class distribution. These biases hinder the generalization ability of PLMs, which is necessary to address many unseen mentions in real-world situations. We propose a novel debiasing method RegLER to improve predictions for entities of varying lengths. To close the gap between evaluation and real-world situations, we evaluated PLMs on partitioned benchmark datasets containing unseen mention sets. Here, RegLER shows significant improvement over long-named entities that can predict through debiasing on conjunction or special characters within entities. Furthermore, there is a severe class imbalance in most NER datasets, causing easy-negative examples to dominate during training, such as "The". Our approach alleviates skewed class distribution by reducing the influence of easy-negative examples. Extensive experiments on the biomedical and general domains demonstrated the generalization capabilities of our method. To facilitate reproducibility and future work, we release our code."https://github.com/minstar/RegLER"