X-RAY: Mapping LLM Reasoning Capability via Formalized and Calibrated Probes
This work provides a more granular understanding of LLM reasoning for researchers and developers, moving beyond task-level accuracy to identify specific structural weaknesses.
This paper introduces X-RAY, a system that maps LLM reasoning capabilities by using formally verified probes to isolate structural information in reasoning problems. Their analysis of state-of-the-art LLMs across various scientific domains reveals that models are robust to constraint refinement but degrade sharply when the solution space's underlying structural form is altered.
Large language models (LLMs) achieve promising performance, yet their ability to reason remains poorly understood. Existing evaluations largely emphasize task-level accuracy, often conflating pattern matching with reasoning capability. We present X-RAY, an explainable reasoning analysis system that maps the LLM reasoning capability using calibrated, formally verified probes. We model reasoning capability as a function of extractable \textit{structure}, operationalized through formal properties such as constraint interaction, reasoning depth, and solution-space geometry. X-Ray generates probes via formal tools with controlled structural variations, enabling precise isolation of incremental structural information through formal calibration and verification. We evaluate state-of-the-art LLMs on problems ranging from junior-level to advanced in mathematics, physics, and chemistry. Our analysis reveals a systematic asymmetry in LLM reasoning: models are relatively robust to constraint refinement, where additional conditions shrink an existing solution space, but degrade sharply under solution-space restructuring, where modifications alter the underlying structural form of the solution manifold. Moreover, calibrated formal probes differentiate models that appear indistinguishable on standard benchmarks and reveal failure modes that are structurally interpretable rather than opaque. Beyond evaluation, our framework is contamination-free and supports the training and testing of reasoning models.