Diverse Inference and Verification for Advanced Reasoning
This addresses the problem of improving reasoning capabilities in LLMs for complex tasks like mathematics and puzzles, though it appears incremental as it builds on existing methods like verification and sampling.
The paper tackles the challenge of advanced reasoning tasks like IMO combinatorics, ARC puzzles, and HLE questions by using a diverse inference approach that combines multiple models and methods at test time, resulting in accuracy improvements from 33.3% to 77.8% on IMO problems, 8% to 37% on HLE, and solving 80% of ARC puzzles unsolved by humans.
Reasoning LLMs such as OpenAI o1, o3 and DeepSeek R1 have made significant progress in mathematics and coding, yet find challenging advanced tasks such as International Mathematical Olympiad (IMO) combinatorics problems, Abstraction and Reasoning Corpus (ARC) puzzles, and Humanity's Last Exam (HLE) questions. We use a diverse inference approach that combines multiple models and methods at test time. We find that verifying mathematics and code problems, and rejection sampling on other problems is simple and effective. We automatically verify correctness of solutions to IMO problems by Lean, and ARC puzzles by code, and find that best-of-N effectively answers HLE questions. Our approach increases answer accuracy on IMO combinatorics problems from 33.3% to 77.8%, accuracy on HLE questions from 8% to 37%, and solves 80% of ARC puzzles that 948 humans could not and 26.5% of ARC puzzles that o3 high compute does not. Test-time simulations, reinforcement learning, and meta-learning with inference feedback improve generalization by adapting agent graph representations and varying prompts, code, and datasets. Our approach is reliable, robust, and scalable, and in the spirit of reproducible research, we will make it publicly available upon publication.