Cascaded Text Generation with Markov Transformers
This addresses the efficiency bottleneck in text generation for applications like machine translation, though it appears incremental as it builds on existing autoregressive and non-autoregressive methods.
The paper tackles the trade-off between generation quality and speed in neural text generation by proposing an autoregressive model with sub-linear parallel time generation. The result is a cascaded decoding approach using Markov transformers that shows competitive accuracy/speed trade-offs on five machine translation datasets.
The two dominant approaches to neural text generation are fully autoregressive models, using serial beam search decoding, and non-autoregressive models, using parallel decoding with no output dependencies. This work proposes an autoregressive model with sub-linear parallel time generation. Noting that conditional random fields with bounded context can be decoded in parallel, we propose an efficient cascaded decoding approach for generating high-quality output. To parameterize this cascade, we introduce a Markov transformer, a variant of the popular fully autoregressive model that allows us to simultaneously decode with specific autoregressive context cutoffs. This approach requires only a small modification from standard autoregressive training, while showing competitive accuracy/speed tradeoff compared to existing methods on five machine translation datasets.