BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMs
This work addresses the issue of factual unreliability in LRMs, which is crucial for users relying on these models for accurate reasoning, though it is an incremental improvement as it builds on existing LRM frameworks.
The paper tackled the problem of overconfident and incorrect answers in Large Reasoning Models (LRMs) by addressing pathological reasoning patterns like last-minute guessing and second-thought spiraling, resulting in an increase in reliability from 39.33% to 61.48% for DeepSeek-R1-Distill-Llama-8B while maintaining comparable accuracy.
Recent advances in Large Reasoning Models (LRMs) have shown impressive capabilities in mathematical and logical reasoning. However, current LRMs rarely admit ignorance or respond with "I don't know". Instead, they often produce incorrect answers while showing undue confidence, raising concerns about their factual reliability. In this work, we identify two pathological reasoning patterns characterized by overthinking that contribute to the overconfident and incorrect answers: last-minute guessing and second-thought spiraling. To address these issues, we propose BARREL-a novel framework that promotes concise and boundary-aware factual reasoning. Our experiments show that BARREL-training increases the reliability of DeepSeek-R1-Distill-Llama-8B from 39.33% to 61.48%, while still achieving accuracy comparable to models finetuned on reasoning data generated by R1. These results demonstrate that our pilot study is inspiring to build more reliable and factual System 2 LRMs.