Character-Aware Decoder for Translation into Morphologically Rich Languages
This addresses the challenge of improving translation quality for morphologically rich languages, which is an incremental advancement in NMT by incorporating character-level information.
The authors tackled the problem of neural machine translation into morphologically rich languages by developing a character-aware decoder that captures lower-level morphological patterns, achieving BLEU score gains of up to +3.05 in low-resource settings across 14 diverse target languages.
Neural machine translation (NMT) systems operate primarily on words (or sub-words), ignoring lower-level patterns of morphology. We present a character-aware decoder designed to capture such patterns when translating into morphologically rich languages. We achieve character-awareness by augmenting both the softmax and embedding layers of an attention-based encoder-decoder model with convolutional neural networks that operate on the spelling of a word. To investigate performance on a wide variety of morphological phenomena, we translate English into 14 typologically diverse target languages using the TED multi-target dataset. In this low-resource setting, the character-aware decoder provides consistent improvements with BLEU score gains of up to $+3.05$. In addition, we analyze the relationship between the gains obtained and properties of the target language and find evidence that our model does indeed exploit morphological patterns.