SymCode: A Neurosymbolic Approach to Mathematical Reasoning via Verifiable Code Generation
This addresses the issue of unverified and arithmetically unsound solutions in mathematical reasoning for AI systems, representing a key step towards more accurate and trustworthy AI in formal domains, though it is an incremental improvement over existing methods.
The paper tackled the problem of unreliable mathematical reasoning in large language models by introducing SymCode, a neurosymbolic framework that generates verifiable code using SymPy, achieving up to 13.6 percentage point accuracy improvements on benchmarks like MATH-500 and OlympiadBench.
Large Language Models (LLMs) often struggle with complex mathematical reasoning, where prose-based generation leads to unverified and arithmetically unsound solutions. Current prompting strategies like Chain of Thought still operate within this unreliable medium, lacking a mechanism for deterministic verification. To address these limitations, we introduce SymCode, a neurosymbolic framework that reframes mathematical problem-solving as a task of verifiable code generation using the SymPy library. We evaluate SymCode on challenging benchmarks, including MATH-500 and OlympiadBench, demonstrating significant accuracy improvements of up to 13.6 percentage points over baselines. Our analysis shows that SymCode is not only more token-efficient but also fundamentally shifts model failures from opaque logical fallacies towards transparent, programmatic errors. By grounding LLM reasoning in a deterministic symbolic engine, SymCode represents a key step towards more accurate and trustworthy AI in formal domains.