Online Versus Offline NMT Quality: An In-depth Analysis on English-German and German-English
This work addresses translation quality issues for users of online neural machine translation systems, particularly in latency-sensitive language pairs like German-English, but it is incremental as it builds on existing models.
The study compared offline and online neural machine translation architectures, specifically convolutional Pervasive Attention and attention-based Transformer models, on English-German and German-English language pairs, finding that online decoding constraints impact translation quality, with results highlighting each model's strengths and weaknesses in online setups.
We conduct in this work an evaluation study comparing offline and online neural machine translation architectures. Two sequence-to-sequence models: convolutional Pervasive Attention (Elbayad et al. 2018) and attention-based Transformer (Vaswani et al. 2017) are considered. We investigate, for both architectures, the impact of online decoding constraints on the translation quality through a carefully designed human evaluation on English-German and German-English language pairs, the latter being particularly sensitive to latency constraints. The evaluation results allow us to identify the strengths and shortcomings of each model when we shift to the online setup.