PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chains
This addresses the problem of improving reasoning accuracy in LLMs for tasks like math and reasoning benchmarks, though it is incremental as it builds on best-of-n sampling with a novel scoring function.
The paper tackles the challenge of selecting correct reasoning chains in large language models without ground-truth answers by proposing PiCSAR, a training-free method that scores candidates using joint log-likelihood, achieving substantial accuracy gains such as +10.18 on MATH500 and +9.81 on AIME2025.
Best-of-n sampling improves the accuracy of large language models (LLMs) and large reasoning models (LRMs) by generating multiple candidate solutions and selecting the one with the highest reward. The key challenge for reasoning tasks is designing a scoring function that can identify correct reasoning chains without access to ground-truth answers. We propose Probabilistic Confidence Selection And Ranking (PiCSAR): a simple, training-free method that scores each candidate generation using the joint log-likelihood of the reasoning and final answer. The joint log-likelihood of the reasoning and final answer naturally decomposes into reasoning confidence and answer confidence. PiCSAR achieves substantial gains across diverse benchmarks (+10.18 on MATH500, +9.81 on AIME2025), outperforming baselines with at least 2x fewer samples in 16 out of 20 comparisons. Our analysis reveals that correct reasoning chains exhibit significantly higher reasoning and answer confidence, justifying the effectiveness of PiCSAR.