treaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding
This work addresses the problem of slow inference in diffusion LLMs for natural language generation, offering a significant speed improvement that is incremental to existing acceleration methods.
The paper tackles inefficiencies in diffusion large language models (dLLMs) by proposing Streaming-dLLM, a training-free framework that accelerates inference via suffix pruning and dynamic decoding, achieving up to 68.2x speedup while maintaining generation quality.
Diffusion Large Language Models (dLLMs) offer a compelling paradigm for natural language generation, leveraging parallel decoding and bidirectional attention to achieve superior global coherence compared to autoregressive models. While recent works have accelerated inference via KV cache reuse or heuristic decoding, they overlook the intrinsic inefficiencies within the block-wise diffusion process. Specifically, they suffer from spatial redundancy by modeling informative-sparse suffix regions uniformly and temporal inefficiency by applying fixed denoising schedules across all the decoding process. To address this, we propose Streaming-dLLM, a training-free framework that streamlines inference across both spatial and temporal dimensions. Spatially, we introduce attenuation guided suffix modeling to approximate the full context by pruning redundant mask tokens. Temporally, we employ a dynamic confidence aware strategy with an early exit mechanism, allowing the model to skip unnecessary iterations for converged tokens. Extensive experiments show that Streaming-dLLM achieves up to 68.2X speedup while maintaining generation quality, highlighting its effectiveness in diffusion decoding. The code is available at https://github.com/xiaoshideta/Streaming-dLLM.