Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-up
This addresses limitations in LLMs' logical capabilities for tasks like mathematical reasoning, though it appears incremental as it builds on existing warm-up and preference-based methods.
The paper tackles the problem of improving large language models' logical reasoning in mathematical and complex tasks by proposing Reversal of Thought (RoT), a plug-and-play framework that uses preference-guided reverse reasoning during warm-up, resulting in enhanced reasoning accuracy and efficiency compared to existing baselines.
Large language models (LLMs) have shown remarkable performance in reasoning tasks but face limitations in mathematical and complex logical reasoning. Existing methods to improve LLMs' logical capabilities either involve traceable or verifiable logical sequences that generate more reliable responses by constructing logical structures yet increase computational costs, or introduces rigid logic template rules, reducing flexibility. In this paper, we propose Reversal of Thought (RoT), a plug-and-play and cost-effective reasoning framework designed to enhance the logical reasoning abilities of LLMs during the warm-up phase prior to batch inference. RoT utilizes a Preference-Guided Reverse Reasoning warm-up strategy, which integrates logical symbols for pseudocode planning through meta-cognitive mechanisms and pairwise preference self-evaluation to generate task-specific prompts solely through demonstrations, aligning with LLMs' cognitive preferences shaped by RLHF. Through reverse reasoning, we utilize a Cognitive Preference Manager to assess knowledge boundaries and further expand LLMs' reasoning capabilities by aggregating solution logic for known tasks and stylistic templates for unknown tasks. Experiments across various tasks demonstrate that RoT surpasses existing baselines in both reasoning accuracy and efficiency.