Duplex Sequence-to-Sequence Learning for Reversible Machine Translation
This work addresses the challenge of reversible machine translation for NLP researchers, offering a novel model that improves performance while being parameter-efficient.
The paper tackled the problem of effectively utilizing bidirectional supervision in sequence-to-sequence learning for machine translation by proposing REDER, a parameter-efficient reversible duplex transformer, which achieved up to 1.3 BLEU improvement over multitask-trained baselines.
Sequence-to-sequence learning naturally has two directions. How to effectively utilize supervision signals from both directions? Existing approaches either require two separate models, or a multitask-learned model but with inferior performance. In this paper, we propose REDER (Reversible Duplex Transformer), a parameter-efficient model and apply it to machine translation. Either end of REDER can simultaneously input and output a distinct language. Thus REDER enables reversible machine translation by simply flipping the input and output ends. Experiments verify that REDER achieves the first success of reversible machine translation, which helps outperform its multitask-trained baselines by up to 1.3 BLEU.