Rectifying Belief Space via Unlearning to Harness LLMs' Reasoning
This addresses the reliability issue in LLMs for users needing accurate reasoning, but it is incremental as it builds on existing unlearning techniques.
The paper tackles the problem of LLMs generating incorrect answers due to spurious beliefs by proposing a method to rectify the belief space through unlearning, which corrects misanswered questions without harming overall performance and improves generalization on unseen data.
Large language models (LLMs) can exhibit advanced reasoning yet still generate incorrect answers. We hypothesize that such errors frequently stem from spurious beliefs, propositions the model internally considers true but are incorrect. To address this, we propose a method to rectify the belief space by suppressing these spurious beliefs while simultaneously enhancing true ones, thereby enabling more reliable inferences. Our approach first identifies the beliefs that lead to incorrect or correct answers by prompting the model to generate textual explanations, using our Forward-Backward Beam Search (FBBS). We then apply unlearning to suppress the identified spurious beliefs and enhance the true ones, effectively rectifying the model's belief space. Empirical results on multiple QA datasets and LLMs show that our method corrects previously misanswered questions without harming overall model performance. Furthermore, our approach yields improved generalization on unseen data, suggesting that rectifying a model's belief space is a promising direction for mitigating errors and enhancing overall reliability.