Fine-Grained Attention Mechanism for Neural Machine Translation
This work addresses a specific bottleneck in neural machine translation for researchers and practitioners, offering an incremental improvement over existing attention methods.
The paper tackles the limitation of existing attention mechanisms in neural machine translation by proposing a fine-grained (2D) attention mechanism that assigns separate attention scores to each dimension of context vectors, resulting in improved translation quality on En-De and En-Fi tasks as measured by BLEU scores.
Neural machine translation (NMT) has been a new paradigm in machine translation, and the attention mechanism has become the dominant approach with the state-of-the-art records in many language pairs. While there are variants of the attention mechanism, all of them use only temporal attention where one scalar value is assigned to one context vector corresponding to a source word. In this paper, we propose a fine-grained (or 2D) attention mechanism where each dimension of a context vector will receive a separate attention score. In experiments with the task of En-De and En-Fi translation, the fine-grained attention method improves the translation quality in terms of BLEU score. In addition, our alignment analysis reveals how the fine-grained attention mechanism exploits the internal structure of context vectors.