Modeling Target-Side Inflection in Neural Machine Translation
This addresses translation quality issues for languages with strong inflection, though it appears incremental as it builds on existing BPE approaches.
The paper tackled the problem of neural machine translation (NMT) struggling with large vocabularies in morphologically rich languages by introducing a method that generates lemmas and POS tags followed by deterministic generation, resulting in improvements for English-Czech and English-German translation.
NMT systems have problems with large vocabulary sizes. Byte-pair encoding (BPE) is a popular approach to solving this problem, but while BPE allows the system to generate any target-side word, it does not enable effective generalization over the rich vocabulary in morphologically rich languages with strong inflectional phenomena. We introduce a simple approach to overcome this problem by training a system to produce the lemma of a word and its morphologically rich POS tag, which is then followed by a deterministic generation step. We apply this strategy for English-Czech and English-German translation scenarios, obtaining improvements in both settings. We furthermore show that the improvement is not due to only adding explicit morphological information.