Watch Your Steps: Observable and Modular Chains of Thought
This work addresses the issue of non-local errors and modularity verification in chain-of-thought learning, offering a method to improve explanation analysis in AI systems.
The authors tackled the problem of making chain-of-thought explanations more observable and modular by introducing Program Trace Prompting, a variant that wraps demonstrations in a formal Python-based syntax, achieving strong results on 23 diverse tasks in the BIG-Bench Hard benchmark.
We propose a variant of chain of thought (CoT) prompting called Program Trace Prompting that makes explanations more observable while preserving the power, generality and flexibility of CoT. In our approach, few-shot CoT demonstrations are wrapped in a formal syntax based on Python, and each prompt: identifies and names steps; defines the input/output behavior of steps; and replaces CoT explanations of in-context examples with chains of these formalized steps on the same examples. Program Trace Prompting is applicable to many tasks, achieving strong results on the 23 diverse tasks in the BIG-Bench Hard benchmark. More importantly, by instrumenting explanations in this way, we enable new types of analysis. In particular, we identify "non-local errors" (which correspond to incorrectly learning the reasoning method illustrated in the demonstrations) as an unaddressed issue in CoT learning, and we present methods for verifying the modularity of steps in a CoT explanation.