Dyve: Thinking Fast and Slow for Dynamic Process Verification
This addresses the problem of improving reasoning accuracy in LLMs for tasks like mathematical problem-solving, though it appears incremental as it builds on existing process verification methods.
The paper tackles reasoning error detection in large language models by introducing Dyve, a dynamic process verifier that integrates fast and slow thinking, which significantly outperforms existing process-based verifiers on ProcessBench and MATH datasets.
We present Dyve, a dynamic process verifier that enhances reasoning error detection in large language models by integrating fast and slow thinking, inspired by Kahneman's Systems Theory. Dyve adaptively applies immediate token-level confirmation System 1 for straightforward steps and comprehensive analysis System 2 for complex ones. Leveraging a novel step-wise consensus-filtered process supervision technique, combining Monte Carlo estimation with LLM based evaluation, Dyve curates high-quality supervision signals from noisy data. Experimental results on ProcessBench and the MATH dataset confirm that Dyve significantly outperforms existing process-based verifiers and boosts performance in Best-of-N settings.