MELM: Data Augmentation with Masked Entity Language Modeling for Low-Resource NER
This addresses data scarcity in low-resource NER, an incremental advance for natural language processing applications.
The paper tackles token-label misalignment in data augmentation for low-resource NER by proposing MELM, a framework that injects NER labels into context to generate high-quality augmented data, resulting in substantial improvements over baselines across monolingual, cross-lingual, and multilingual settings.
Data augmentation is an effective solution to data scarcity in low-resource scenarios. However, when applied to token-level tasks such as NER, data augmentation methods often suffer from token-label misalignment, which leads to unsatsifactory performance. In this work, we propose Masked Entity Language Modeling (MELM) as a novel data augmentation framework for low-resource NER. To alleviate the token-label misalignment issue, we explicitly inject NER labels into sentence context, and thus the fine-tuned MELM is able to predict masked entity tokens by explicitly conditioning on their labels. Thereby, MELM generates high-quality augmented data with novel entities, which provides rich entity regularity knowledge and boosts NER performance. When training data from multiple languages are available, we also integrate MELM with code-mixing for further improvement. We demonstrate the effectiveness of MELM on monolingual, cross-lingual and multilingual NER across various low-resource levels. Experimental results show that our MELM presents substantial improvement over the baseline methods.