Meta-Reasoning: Semantics-Symbol Deconstruction for Large Language Models
This work addresses the problem of making symbolic methods more applicable and adaptable to real-world reasoning tasks for users of large language models, though it appears incremental as it builds on existing neural-symbolic approaches.
The authors tackled the limitation of existing neural-symbolic methods that require converting reasoning tasks into programs, which deviates from human reasoning habits, by proposing Meta-Reasoning to deconstruct semantic information into generic symbolic representations, resulting in significant improvements in in-context reasoning accuracy, learning efficiency, out-of-domain generalization, and output stability compared to Chain-of-Thought across over ten datasets.
Neural-symbolic methods have demonstrated efficiency in enhancing the reasoning abilities of large language models (LLMs). However, existing methods mainly rely on syntactically mapping natural languages to complete formal languages like Python and SQL. Those methods require that reasoning tasks be convertible into programs, which cater to the computer execution mindset and deviate from human reasoning habits. To broaden symbolic methods' applicability and adaptability in the real world, we propose the Meta-Reasoning from a linguistic perspective. This method empowers LLMs to deconstruct reasoning-independent semantic information into generic symbolic representations, thereby efficiently capturing more generalized reasoning knowledge. We conduct extensive experiments on more than ten datasets encompassing conventional reasoning tasks like arithmetic, symbolic, and logical reasoning, and the more complex interactive reasoning tasks like theory-of-mind reasoning. Experimental results demonstrate that Meta-Reasoning significantly enhances in-context reasoning accuracy, learning efficiency, out-of-domain generalization, and output stability compared to the Chain-of-Thought technique. Code and data are publicly available at \url{https://github.com/Alsace08/Meta-Reasoning}.