Speculative Beam Search for Simultaneous Translation
This addresses the problem of efficient and accurate real-time translation for users of simultaneous translation systems, representing a novel method for a known bottleneck.
The paper tackles the challenge of applying beam search to simultaneous translation, where words are committed on the fly, by proposing a speculative beam search algorithm that hallucinates future steps to improve decision-making. Experiments show large improvements over previous work across diverse language pairs.
Beam search is universally used in full-sentence translation but its application to simultaneous translation remains non-trivial, where output words are committed on the fly. In particular, the recently proposed wait-k policy (Ma et al., 2019a) is a simple and effective method that (after an initial wait) commits one output word on receiving each input word, making beam search seemingly impossible. To address this challenge, we propose a speculative beam search algorithm that hallucinates several steps into the future in order to reach a more accurate decision, implicitly benefiting from a target language model. This makes beam search applicable for the first time to the generation of a single word in each step. Experiments over diverse language pairs show large improvements over previous work.