Advancing Semi-Supervised Learning for Automatic Post-Editing: Data-Synthesis by Mask-Infilling with Erroneous Terms
This work addresses the data scarcity problem for researchers and practitioners in machine translation by improving semi-supervised learning for APE, though it is incremental as it builds on existing noising-based methods.
The paper tackles the lack of training data for automatic post-editing (APE) models by proposing a data-synthesis method that mimics translation errors using mask-infilling with erroneous terms and selective corpus interleaving, resulting in significantly better APE performance compared to existing methods.
Semi-supervised learning that leverages synthetic data for training has been widely adopted for developing automatic post-editing (APE) models due to the lack of training data. With this aim, we focus on data-synthesis methods to create high-quality synthetic data. Given that APE takes as input a machine-translation result that might include errors, we present a data-synthesis method by which the resulting synthetic data mimic the translation errors found in actual data. We introduce a noising-based data-synthesis method by adapting the masked language model approach, generating a noisy text from a clean text by infilling masked tokens with erroneous tokens. Moreover, we propose selective corpus interleaving that combines two separate synthetic datasets by taking only the advantageous samples to enhance the quality of the synthetic data further. Experimental results show that using the synthetic data created by our approach results in significantly better APE performance than other synthetic data created by existing methods.