Diffuse Thinking: Exploring Diffusion Language Models as Efficient Thought Proposers for Reasoning
This work addresses the computational burden in reasoning tasks for AI researchers and practitioners, offering an incremental improvement by combining existing model types.
The paper tackles the inefficiency of autoregressive large language models (LLMs) in generating intermediate thoughts for reasoning by proposing a collaborative framework that uses diffusion language models (DLMs) to efficiently propose candidate thoughts and LLMs to evaluate them, achieving strong performance on complex reasoning benchmarks.
In recent years, large language models (LLMs) have witnessed remarkable advancements, with the test-time scaling law consistently enhancing the reasoning capabilities. Through systematic evaluation and exploration of a diverse spectrum of intermediate thoughts, LLMs demonstrate the potential to generate deliberate reasoning steps, thereby substantially enhancing reasoning accuracy. However, LLMs' autoregressive generation paradigm results in reasoning performance scaling sub-optimally with test-time computation, often requiring excessive computational overhead to propose thoughts while yielding only marginal performance gains. In contrast, diffusion language models (DLMs) can efficiently produce diverse samples through parallel denoising in a single forward pass, inspiring us to leverage them for proposing intermediate thoughts, thereby alleviating the computational burden associated with autoregressive generation while maintaining quality. In this work, we propose an efficient collaborative reasoning framework, leveraging DLMs to generate candidate thoughts and LLMs to evaluate their quality. Experiments across diverse benchmarks demonstrate that our framework achieves strong performance in complex reasoning tasks, offering a promising direction for future research. Our code is open-source at https://anonymous.4open.science/r/Diffuse-Thinking-EC60.