An Effective Incorporating Heterogeneous Knowledge Curriculum Learning for Sequence Labeling
This work addresses the slow training problem in sequence labeling for natural language processing, but it is incremental as it builds on existing curriculum learning methods.
The paper tackles the challenge of incorporating external knowledge into sequence labeling models, which introduces data heterogeneity and complexity, by proposing a two-stage curriculum learning framework that improves performance and training speed, as demonstrated on six Chinese datasets.
Sequence labeling models often benefit from incorporating external knowledge. However, this practice introduces data heterogeneity and complicates the model with additional modules, leading to increased expenses for training a high-performing model. To address this challenge, we propose a two-stage curriculum learning (TCL) framework specifically designed for sequence labeling tasks. The TCL framework enhances training by gradually introducing data instances from easy to hard, aiming to improve both performance and training speed. Furthermore, we explore different metrics for assessing the difficulty levels of sequence labeling tasks. Through extensive experimentation on six Chinese word segmentation (CWS) and Part-of-speech tagging (POS) datasets, we demonstrate the effectiveness of our model in enhancing the performance of sequence labeling models. Additionally, our analysis indicates that TCL accelerates training and alleviates the slow training problem associated with complex models.