Systematic Diagnosis of Brittle Reasoning in Large Language Models
This provides a more granular evaluation method for mathematical comprehension in AI, offering a roadmap for developing more reliable applications, though it is incremental in focusing on diagnosis rather than solving the brittleness.
The paper tackles the problem of measuring mathematical reasoning in large language models by diagnosing specific failure points, revealing that while models achieve near-perfect accuracy on procedural tasks, performance drops sharply on combinatorial reasoning with restrictions.
A central question in artificial intelligence is the extent to which machine learning models comprehend mathematics. To address this, we propose a novel framework for measuring mathematical reasoning that moves beyond standard benchmarks to diagnose specific failure points. Our method first generates structured, step-by-step reasoning from gpt-3.5-turbo on the GSM8K dataset. We then use a more capable analyst model, gpt-4o-mini, to categorize errors and, crucially, perform an unsupervised clustering of every reasoning sentence to identify emergent "reasoning modes." This analysis reveals a cognitive profile with a stark, nonhuman-like brittleness: while the model achieves near-perfect accuracy on procedural modes like sequential calculation, its performance on modes requiring combinatorial reasoning with restrictions plummets. By identifying and quantifying the reliability of these distinct reasoning skills, our work provides a more granular method to evaluate mathematical comprehension and offers a precise roadmap for developing new capabilities and more reliable future applications.