HYBRIDMIND: Meta Selection of Natural Language and Symbolic Language for Enhanced LLM Reasoning
This addresses the challenge of improving reasoning accuracy in LLMs for tasks like logical and mathematical problem-solving, though it is incremental as it builds on existing methods.
The paper tackles the problem of enhancing LLM reasoning by selecting between natural and symbolic languages, finding that fine-tuning LLaMA-3.1-8B-Instruct as a meta-selector outperforms GPT-4o by 4.4% on FOLIO and 1.3% on MATH, and using GPT-3.5-turbo as a prompted meta-selector yields a 10% improvement on FOLIO's challenging subset.
LLMs approach logical and mathematical reasoning through natural or symbolic languages. While natural language offers human-accessible flexibility but suffers from ambiguity, symbolic reasoning provides precise, machine-executable inferences at the cost of strict domain constraints. We introduce HYBRIDMIND, an adaptive strategy that selects the optimal reasoning approach for each reasoning problem. Through extensive experiments, we evaluate both prompting-based approaches with state-of-the-art LLMs and fine-tuned open-source models. We find that fine-tuning LLaMA-3.1-8B-Instruct as a meta-selector outperforms GPT-4o's natural language reasoning by 4.4\% on FOLIO and 1.3\% on MATH. More notably, using GPT-3.5-turbo as a prompted meta-selector yields a 10\% improvement on FOLIO's challenging subset compared to GPT-4o. We will release our code and data to support future research.