Amortized Noisy Channel Neural Machine Translation
This work addresses the inference speed bottleneck for noisy channel NMT, making it more feasible for real-world applications, though it is incremental as it builds on existing noisy channel models.
The paper tackled the computational inefficiency of noisy channel neural machine translation by developing amortized models that use greedy decoding, achieving similar translation quality (BLEU/BLEURT) to slower beam search and rerank methods while speeding up inference by 1-2 orders of magnitude.
Noisy channel models have been especially effective in neural machine translation (NMT). However, recent approaches like "beam search and rerank" (BSR) incur significant computation overhead during inference, making real-world application infeasible. We aim to study if it is possible to build an amortized noisy channel NMT model such that when we do greedy decoding during inference, the translation accuracy matches that of BSR in terms of reward (based on the source-to-target log probability and the target-to-source log probability) and quality (based on BLEU and BLEURT). We attempt three approaches to train the new model: knowledge distillation, one-step-deviation imitation learning, and Q learning. The first approach obtains the noisy channel signal from a pseudo-corpus, and the latter two approaches aim to optimize toward a noisy-channel MT reward directly. For all three approaches, the generated translations fail to achieve rewards comparable to BSR, but the translation quality approximated by BLEU and BLEURT is similar to the quality of BSR-produced translations. Additionally, all three approaches speed up inference by 1-2 orders of magnitude.