Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning
This addresses inference speed and fluency issues in diffusion LMs for text generation, though it is incremental as it builds on existing diffusion LM frameworks.
The paper tackled the long decoding-window problem in diffusion language models, which causes irrelevant or repetitive tokens, by proposing convolutional decoding and rejective fine-tuning, achieving state-of-the-art results on benchmarks like AlpacaEval with significantly lower step size.
Autoregressive (AR) language models generate text one token at a time, which limits their inference speed. Diffusion-based language models offer a promising alternative, as they can decode multiple tokens in parallel. However, we identify a key bottleneck in current diffusion LMs: the long decoding-window problem, where tokens generated far from the input context often become irrelevant or repetitive. Previous solutions like semi-autoregressive address this issue by splitting windows into blocks (sacrificing bidirectionality), but we find that this also leads to time-interval expansion problem, sacrificing the speed. Therefore, semi-AR eliminates the main advantages of diffusion models. To overcome this, we propose Convolutional decoding (Conv), a normalization-based method that narrows the decoding window without hard segmentation, leading to better fluency and flexibility. Additionally, we introduce Rejecting Rule-based Fine-Tuning (R2FT), a post-hoc training scheme that better aligns tokens at positions far from context. Our methods achieve state-of-the-art results on open-ended generation benchmarks (e.g., AlpacaEval) among diffusion LM baselines, with significantly lower step size than previous works, demonstrating both speed and quality improvements.