Tagged Back-Translation
This work addresses efficiency and performance in machine translation for researchers and practitioners, offering a simpler method to enhance back-translation.
The paper tackled the problem of improving Neural Machine Translation by showing that synthetic noise in back-translation primarily indicates synthetic data rather than diversifying it, and proposed tagging with an extra token as a simpler alternative, achieving state-of-the-art results on WMT English-Romanian and matching performance on English-German.
Recent work in Neural Machine Translation (NMT) has shown significant quality gains from noised-beam decoding during back-translation, a method to generate synthetic parallel data. We show that the main role of such synthetic noise is not to diversify the source side, as previously suggested, but simply to indicate to the model that the given source is synthetic. We propose a simpler alternative to noising techniques, consisting of tagging back-translated source sentences with an extra token. Our results on WMT outperform noised back-translation in English-Romanian and match performance on English-German, re-defining state-of-the-art in the former.