Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement
This work addresses the decoding speed bottleneck for sequence generation tasks like machine translation and image captioning, offering a practical improvement over autoregressive methods.
The authors tackled the problem of slow decoding in neural sequence generation by proposing a deterministic non-autoregressive model based on iterative refinement, which significantly speeds up decoding while maintaining generation quality comparable to autoregressive models, as evaluated on machine translation and image caption tasks.
We propose a conditional non-autoregressive neural sequence model based on iterative refinement. The proposed model is designed based on the principles of latent variable models and denoising autoencoders, and is generally applicable to any sequence generation task. We extensively evaluate the proposed model on machine translation (En-De and En-Ro) and image caption generation, and observe that it significantly speeds up decoding while maintaining the generation quality comparable to the autoregressive counterpart.