The Shape of Reasoning: Topological Analysis of Reasoning Traces in Large Language Models
This work addresses the challenge of automated and efficient evaluation of reasoning quality in large language models, offering a practical tool for researchers and developers, though it is incremental as it builds on existing topological methods applied to a new domain.
The paper tackled the problem of evaluating reasoning traces in large language models, which is currently labor-intensive and unreliable, by introducing a topological data analysis framework that captures geometric structures; the result showed that topological features provided substantially higher predictive power for assessing reasoning quality compared to standard graph metrics.
Evaluating the quality of reasoning traces from large language models remains understudied, labor-intensive, and unreliable: current practice relies on expert rubrics, manual annotation, and slow pairwise judgments. Automated efforts are dominated by graph-based proxies that quantify structural connectivity but do not clarify what constitutes high-quality reasoning; such abstractions can be overly simplistic for inherently complex processes. We introduce a topological data analysis (TDA)-based evaluation framework that captures the geometry of reasoning traces and enables label-efficient, automated assessment. In our empirical study, topological features yield substantially higher predictive power for assessing reasoning quality than standard graph metrics, suggesting that effective reasoning is better captured by higher-dimensional geometric structures rather than purely relational graphs. We further show that a compact, stable set of topological features reliably indicates trace quality, offering a practical signal for future reinforcement learning algorithms.