A Theoretical Understanding of Chain-of-Thought: Coherent Reasoning and Error-Aware Demonstration
This work provides incremental theoretical insights into CoT prompting for improving reasoning in large language models, addressing error sensitivity in intermediate steps.
The paper tackles the theoretical understanding of Chain-of-Thought (CoT) prompting by showing that integrating coherent reasoning improves error correction and prediction accuracy in transformers compared to stepwise methods, and proposes an enhanced CoT approach using both correct and incorrect reasoning paths in demonstrations.
Few-shot Chain-of-Thought (CoT) prompting has demonstrated strong performance in improving the reasoning capabilities of large language models (LLMs). While theoretical investigations have been conducted to understand CoT, the underlying transformer used in these studies isolates the CoT reasoning process into separated in-context learning steps (Stepwise ICL). In this work, we theoretically show that, compared to Stepwise ICL, the transformer gains better error correction ability and more accurate predictions if the reasoning from earlier steps (Coherent CoT) is integrated. Given that this coherent reasoning changes the behavior of the transformer, we further investigate the sensitivity of the transformer with Coherent CoT when the demonstration examples are corrupted at the inference stage. Our theoretical results indicate that the transformer is more sensitive to errors in intermediate reasoning steps than the final outcome. Building upon this observation, we propose an improvement on CoT by incorporating both correct and incorrect reasoning paths in the demonstration. Our experiments validate the effectiveness of the proposed approach.