FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning
This addresses the problem of evaluating transparent and verifiable financial reasoning for AI developers, though it's incremental as it builds on existing Chain-of-Thought evaluation concepts.
The authors tackled the lack of benchmarks for verifiable multi-step reasoning in finance by introducing FinChain, a symbolic benchmark with 58 topics across 12 domains that enables machine-verifiable reasoning through executable Python traces, and found that even frontier LLMs show limitations while domain-adapted models narrow the gap.
Multi-step symbolic reasoning is essential for robust financial analysis; yet, current benchmarks largely overlook this capability. Existing datasets such as FinQA and ConvFinQA emphasize final numerical answers while neglecting the intermediate reasoning required for transparency and verification. To address this gap, we introduce FinChain, the first benchmark specifically designed for verifiable Chain-of-Thought (CoT) evaluation in finance. FinChain spans 58 topics across 12 financial domains, each represented by parameterized symbolic templates with executable Python traces that enable fully machine-verifiable reasoning and scalable, contamination-free data generation. To assess reasoning capacity, we propose ChainEval, a dynamic alignment metric that jointly evaluates both the final-answer correctness and the step-level reasoning consistency. Evaluating 26 leading LLMs reveals that even frontier proprietary systems exhibit clear limitations in symbolic financial reasoning, while domain-adapted and math-enhanced fine-tuned models substantially narrow this gap. Overall, FinChain exposes persistent weaknesses in multi-step financial reasoning and provides a foundation for developing trustworthy, interpretable, and verifiable financial AI.