Towards Two-Dimensional Sequence to Sequence Model in Neural Machine Translation
This work addresses translation accuracy for NLP researchers and practitioners, but it is incremental as it builds on existing sequence-to-sequence models.
The paper tackles the problem of neural machine translation by proposing a two-dimensional mapping approach using multi-dimensional LSTM, which consistently improves over attention-based sequence-to-sequence models on WMT 2017 German-English tasks.
This work investigates an alternative model for neural machine translation (NMT) and proposes a novel architecture, where we employ a multi-dimensional long short-term memory (MDLSTM) for translation modeling. In the state-of-the-art methods, source and target sentences are treated as one-dimensional sequences over time, while we view translation as a two-dimensional (2D) mapping using an MDLSTM layer to define the correspondence between source and target words. We extend beyond the current sequence to sequence backbone NMT models to a 2D structure in which the source and target sentences are aligned with each other in a 2D grid. Our proposed topology shows consistent improvements over attention-based sequence to sequence model on two WMT 2017 tasks, German$\leftrightarrow$English.