Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models
This addresses the problem of improving spelling correction accuracy for Chinese language users, but it is incremental as it builds on existing sequence-to-sequence frameworks.
The paper tackles the problem of Chinese Spelling Correction (CSC) models failing to correct errors covered by confusion sets and encountering unseen errors, by proposing a method that identifies weak spots to generate adversarial examples and uses task-specific pre-training, resulting in state-of-the-art performance across three datasets.
A sequence-to-sequence learning with neural networks has empirically proven to be an effective framework for Chinese Spelling Correction (CSC), which takes a sentence with some spelling errors as input and outputs the corrected one. However, CSC models may fail to correct spelling errors covered by the confusion sets, and also will encounter unseen ones. We propose a method, which continually identifies the weak spots of a model to generate more valuable training instances, and apply a task-specific pre-training strategy to enhance the model. The generated adversarial examples are gradually added to the training set. Experimental results show that such an adversarial training method combined with the pretraining strategy can improve both the generalization and robustness of multiple CSC models across three different datasets, achieving stateof-the-art performance for CSC task.