Advanced Financial Reasoning at Scale: A Comprehensive Evaluation of Large Language Models on CFA Level III
This provides critical guidance for financial institutions on model selection for high-stakes applications, though it is incremental as it focuses on evaluation rather than new methods.
The paper tackled the problem of evaluating large language models (LLMs) for advanced financial reasoning by benchmarking 23 state-of-the-art models on the CFA Level III exam, achieving composite scores up to 79.1% with leading models like o4-mini and Gemini 2.5 Flash.
As financial institutions increasingly adopt Large Language Models (LLMs), rigorous domain-specific evaluation becomes critical for responsible deployment. This paper presents a comprehensive benchmark evaluating 23 state-of-the-art LLMs on the Chartered Financial Analyst (CFA) Level III exam - the gold standard for advanced financial reasoning. We assess both multiple-choice questions (MCQs) and essay-style responses using multiple prompting strategies including Chain-of-Thought and Self-Discover. Our evaluation reveals that leading models demonstrate strong capabilities, with composite scores such as 79.1% (o4-mini) and 77.3% (Gemini 2.5 Flash) on CFA Level III. These results, achieved under a revised, stricter essay grading methodology, indicate significant progress in LLM capabilities for high-stakes financial applications. Our findings provide crucial guidance for practitioners on model selection and highlight remaining challenges in cost-effective deployment and the need for nuanced interpretation of performance against professional benchmarks.