Reasoning Beyond the Obvious: Evaluating Divergent and Convergent Thinking in LLMs for Financial Scenarios
This addresses the need for better reasoning benchmarks in finance to assess LLMs for safe and strategic deployment, though it is incremental as it builds on existing evaluation frameworks.
The authors tackled the problem of evaluating both divergent and convergent thinking in LLMs for financial scenarios, introducing the ConDiFi benchmark with 607 prompts for divergent reasoning and 990 MCQs for convergent reasoning, and found that GPT-4o underperforms on Novelty and Actionability while models like DeepSeek-R1 and Cohere Command R+ excel in generating actionable insights.
Most reasoning benchmarks for LLMs emphasize factual accuracy or step-by-step logic. In finance, however, professionals must not only converge on optimal decisions but also generate creative, plausible futures under uncertainty. We introduce ConDiFi, a benchmark that jointly evaluates divergent and convergent thinking in LLMs for financial tasks. ConDiFi features 607 macro-financial prompts for divergent reasoning and 990 multi-hop adversarial MCQs for convergent reasoning. Using this benchmark, we evaluated 14 leading models and uncovered striking differences. Despite high fluency, GPT-4o underperforms on Novelty and Actionability. In contrast, models like DeepSeek-R1 and Cohere Command R+ rank among the top for generating actionable, insights suitable for investment decisions. ConDiFi provides a new perspective to assess reasoning capabilities essential to safe and strategic deployment of LLMs in finance.