Towards Generalizable and Faithful Logic Reasoning over Natural Language via Resolution Refutation
This addresses the issue of incomplete logic reasoning in LLMs for natural language processing, offering a more generalizable and faithful solution, though it is incremental as it builds on existing resolution refutation methods.
The paper tackles the problem of first-order logic reasoning over natural language, where previous LLM-based systems are theoretically incomplete and limited in generalization. The proposed GFaiR framework introduces resolution refutation, achieving state-of-the-art performance in complex scenarios while maintaining performance in simple ones.
Large language models (LLMs) have achieved significant performance in various natural language reasoning tasks. However, they still struggle with performing first-order logic reasoning over formal logical theories expressed in natural language. This is because the previous LLMs-based reasoning systems have the theoretical incompleteness issue. As a result, it can only address a limited set of simple reasoning problems, which significantly decreases their generalization ability. To address this issue, we propose a novel framework, named Generalizable and Faithful Reasoner (GFaiR), which introduces the paradigm of resolution refutation. Resolution refutation has the capability to solve all first-order logic reasoning problems by extending reasoning rules and employing the principle of proof by contradiction, so our system's completeness can be improved by introducing resolution refutation. Experimental results demonstrate that our system outperforms previous works by achieving state-of-the-art performances in complex scenarios while maintaining performances in simple scenarios. Besides, we observe that GFaiR is faithful to its reasoning process.