BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving
For practitioners deploying diffusion LLMs, BlockServe significantly improves throughput by mitigating straggler-induced inefficiencies.
BlockServe addresses convergence heterogeneity in diffusion LLM serving by introducing block-grained continuous batching, achieving 1.9–10.6× throughput over Fast-dLLM across five benchmarks.
Efficient serving of diffusion large language models (dLLMs) is hindered by convergence heterogeneity: when batching multiple requests, different sequences converge at different rates, causing faster requests to stall behind slower stragglers and introducing compute bubbles and tail latency. We present BlockServe, a continuous batching framework that integrates block-grained scheduling -- immediately evicting completed requests at block boundaries -- with mixed-state execution that extends dual cache and parallel decoding to heterogeneous batches via gather-scatter indexing. Furthermore, a compute-aware admission controller expands effective batch capacity through token-budgeted refill. On Dream and LLaDA across five benchmarks, BlockServe achieves 1.9--10.6$\times$ throughput over Fast-dLLM with comparable generation quality, establishing block-grained scheduling as a foundation for high-throughput offline dLLM inference.