Attention Strategies for Multi-Source Sequence-to-Sequence Learning
This addresses a relatively unexplored area for tasks involving multiple source languages or modalities, but it is incremental as it builds on existing attention techniques.
The paper tackled the problem of modeling attention in multi-source sequence-to-sequence learning, proposing flat and hierarchical combination methods, and achieved competitive results on WMT16 Multimodal Translation and Automatic Post-editing tasks.
Modeling attention in neural multi-source sequence-to-sequence learning remains a relatively unexplored area, despite its usefulness in tasks that incorporate multiple source languages or modalities. We propose two novel approaches to combine the outputs of attention mechanisms over each source sequence, flat and hierarchical. We compare the proposed methods with existing techniques and present results of systematic evaluation of those methods on the WMT16 Multimodal Translation and Automatic Post-editing tasks. We show that the proposed methods achieve competitive results on both tasks.