Decomposing Reasoning Efficiency in Large Language Models
This work addresses the need for better efficiency metrics in AI reasoning tasks, offering a method to identify bottlenecks without human labeling, but it is incremental as it builds on existing evaluation frameworks.
The authors tackled the problem of evaluating reasoning efficiency in large language models by introducing a framework that decomposes token efficiency into interpretable factors, revealing that accuracy and token-efficiency rankings diverge with a Spearman correlation of 0.63 and verbalization overhead varies by about 9 times across models.
Large language models trained for reasoning trade off inference tokens against accuracy, yet standard evaluations report only final accuracy, obscuring where tokens are spent or wasted. We introduce a trace-optional framework that decomposes token efficiency into interpretable factors: completion under a fixed token budget (avoiding truncation), conditional correctness given completion, and verbosity (token usage). When benchmark metadata provides per-instance workload proxies, we further factor verbosity into two components: mean verbalization overhead (tokens per work unit) and a coupling coefficient capturing how overhead scales with task workload. When reasoning traces are available, we add deterministic trace-quality measures (grounding, repetition, prompt copying) to separate degenerate looping from verbose-but-engaged reasoning, avoiding human labeling and LLM judges. Evaluating 25 models on CogniLoad, we find that accuracy and token-efficiency rankings diverge (Spearman $ρ=0.63$), efficiency gaps are often driven by conditional correctness, and verbalization overhead varies by about 9 times (only weakly related to model scale). Our decomposition reveals distinct bottleneck profiles that suggest different efficiency interventions.