Efficient Wait-k Models for Simultaneous Machine Translation
This work addresses efficient real-time translation for spoken language applications, but it is incremental as it builds on existing wait-k methods with specific optimizations.
The paper tackles the problem of simultaneous machine translation in low-resource spoken language settings by improving wait-k decoder training with unidirectional encoders and multi-k training, achieving competitive performance across various latency levels and showing that 2D-convolutional architectures can match Transformers for this task.
Simultaneous machine translation consists in starting output generation before the entire input sequence is available. Wait-k decoders offer a simple but efficient approach for this problem. They first read k source tokens, after which they alternate between producing a target token and reading another source token. We investigate the behavior of wait-k decoding in low resource settings for spoken corpora using IWSLT datasets. We improve training of these models using unidirectional encoders, and training across multiple values of k. Experiments with Transformer and 2D-convolutional architectures show that our wait-k models generalize well across a wide range of latency levels. We also show that the 2D-convolution architecture is competitive with Transformers for simultaneous translation of spoken language.