Large Language Models as Analogical Reasoners
This addresses the problem of reducing manual effort and improving adaptability in reasoning tasks for users of large language models, though it is incremental as it builds on existing CoT methods.
The paper tackles the need for labeled exemplars in chain-of-thought prompting by introducing analogical prompting, which enables large language models to self-generate relevant exemplars before solving problems, resulting in improved performance over 0-shot and manual few-shot CoT across tasks like GSM8K, MATH, Codeforces, and BIG-Bench.
Chain-of-thought (CoT) prompting for language models demonstrates impressive performance across reasoning tasks, but typically needs labeled exemplars of the reasoning process. In this work, we introduce a new prompting approach, analogical prompting, designed to automatically guide the reasoning process of large language models. Inspired by analogical reasoning, a cognitive process in which humans draw from relevant past experiences to tackle new problems, our approach prompts language models to self-generate relevant exemplars or knowledge in the context, before proceeding to solve the given problem. This method presents several advantages: it obviates the need for labeling or retrieving exemplars, offering generality and convenience; it can also tailor the generated exemplars and knowledge to each problem, offering adaptability. Experimental results show that our approach outperforms 0-shot CoT and manual few-shot CoT in a variety of reasoning tasks, including math problem solving in GSM8K and MATH, code generation in Codeforces, and other reasoning tasks in BIG-Bench.