Character-Level Translation with Self-attention
This work addresses translation efficiency and robustness for multilingual systems, though it is incremental as it modifies an existing model.
The paper tackles character-level neural machine translation by testing a standard transformer and a novel variant with convolutional encoder blocks, finding that the variant consistently outperforms the standard transformer, converges faster, and learns more robust alignments on datasets like WMT and UN.
We explore the suitability of self-attention models for character-level neural machine translation. We test the standard transformer model, as well as a novel variant in which the encoder block combines information from nearby characters using convolutions. We perform extensive experiments on WMT and UN datasets, testing both bilingual and multilingual translation to English using up to three input languages (French, Spanish, and Chinese). Our transformer variant consistently outperforms the standard transformer at the character-level and converges faster while learning more robust character-level alignments.