d$^2$Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
This work addresses inference efficiency for users of diffusion-based LLMs, offering an incremental improvement through a novel caching method.
The paper tackles the problem of inefficient inference in diffusion-based large language models (dLLMs) by introducing d$^2$Cache, a training-free KV cache framework that accelerates inference and improves generation quality, achieving substantial speedups and consistent gains in experiments on models like LLaDA and Dream.
Diffusion-based large language models (dLLMs), despite their promising performance, still suffer from inferior inference efficiency. This is because dLLMs rely on bidirectional attention and cannot directly benefit from the standard key-value (KV) cache as autoregressive models (ARMs) do. To tackle this issue, we introduce \textit{Dual aDaptive Cache} (d$^2$Cache), which is a training-free approximate KV cache framework for accelerating dLLM inference. d$^2$Cache features a two-stage fine-grained selection strategy to identify tokens and adaptively update their KV states at each decoding step, while caching the KV states of the remaining tokens for reuse. Furthermore, d$^2$Cache naturally offers a more reliable decoding alternative, which can enable quasi left-to-right generation and mitigate premature overconfidence in tokens at the end of the sequence. Extensive experimental results on two representative dLLMs (\ie, LLaDA and Dream) demonstrate that d$^2$Cache not only achieves substantial inference speedups, but also yields consistent improvements in generation quality. The code is available at https://github.com/Kamichanw/d2Cache.