A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models
For researchers and practitioners using diffusion language models, this work provides a simple, efficient way to improve sample diversity, which is crucial for tasks like text generation and post-training scaling.
This work addresses the lack of diversity in sampling from diffusion language models (dLLMs) by introducing tempered remasking heuristics that increase diversity without sacrificing speed or quality. The proposed method closes the exploration gap (pass@k) between confidence-based and autoregressive sampling, outperforming both when controlling for cost (pass@NFE).
Much work has been done on designing fast and accurate sampling for diffusion language models (dLLMs). However, these efforts have largely focused on the tradeoff between speed and quality of individual samples; how to additionally ensure diversity across samples remains less well understood. In this work, we show that diversity can be increased by using softened, tempered versions of familiar confidence-based remasking heuristics, retaining their computational benefits and offering simple implementations. We motivate this approach by introducing an idealized formal model of fork tokens and studying the impact of remasking on the expected entropy at the forks. Empirically, the proposed tempered heuristics close the exploration gap (pass@k) between existing confidence-based and autoregressive sampling, hence outperforming both when controlling for cost (pass@NFE). We further study how the increase in diversity translates to downstream post-training and test-time compute scaling. Overall, our findings demonstrate that simple, efficient, and diverse sampling from dLLMs is possible.