Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles
This work addresses inference latency issues for users of diffusion-based language models, offering a significant speed improvement while maintaining quality, though it is incremental as it builds on existing sampling methods.
The paper tackles the problem of inefficient sampling in diffusion-based language models by proposing SlowFast Sampling, a dynamic strategy that adaptively alternates between exploratory and accelerated decoding stages, achieving up to 15.63x speedup on LLaDA with minimal accuracy drop and up to 34.22x when combined with caching.
Diffusion-based language models (dLLMs) have emerged as a promising alternative to traditional autoregressive LLMs by enabling parallel token generation and significantly reducing inference latency. However, existing sampling strategies for dLLMs, such as confidence-based or semi-autoregressive decoding, often suffer from static behavior, leading to suboptimal efficiency and limited flexibility. In this paper, we propose SlowFast Sampling, a novel dynamic sampling strategy that adaptively alternates between exploratory and accelerated decoding stages. Our method is guided by three golden principles: certainty principle, convergence principle, and positional principle, which govern when and where tokens can be confidently and efficiently decoded. We further integrate our strategy with dLLM-Cache to reduce redundant computation. Extensive experiments across benchmarks and models show that SlowFast Sampling achieves up to 15.63$\times$ speedup on LLaDA with minimal accuracy drop, and up to 34.22$\times$ when combined with caching. Notably, our approach outperforms strong autoregressive baselines like LLaMA3 8B in throughput, demonstrating that well-designed sampling can unlock the full potential of dLLMs for fast and high-quality generation.