Towards Robust Neural Machine Translation
This addresses robustness issues in neural machine translation, which is crucial for reliable real-world applications, though it appears incremental as it builds on existing adversarial training methods.
The paper tackles the problem of neural machine translation models being sensitive to small input perturbations by proposing adversarial stability training to make encoders and decoders behave similarly for original and perturbed inputs, achieving significant improvements on Chinese-English, English-German, and English-French translation tasks.
Small perturbations in the input can severely distort intermediate representations and thus impact translation quality of neural machine translation (NMT) models. In this paper, we propose to improve the robustness of NMT models with adversarial stability training. The basic idea is to make both the encoder and decoder in NMT models robust against input perturbations by enabling them to behave similarly for the original input and its perturbed counterpart. Experimental results on Chinese-English, English-German and English-French translation tasks show that our approaches can not only achieve significant improvements over strong NMT systems but also improve the robustness of NMT models.