CDW-CoT: Clustered Distance-Weighted Chain-of-Thoughts Reasoning
This addresses the need for more adaptive reasoning methods in LLMs for diverse datasets, representing an incremental improvement over existing CoT techniques.
The paper tackles the problem of one-size-fits-all Chain of Thought prompting in LLMs by proposing CDW-CoT, which dynamically constructs tailored prompts based on data clustering, resulting in average accuracy improvements of 25.34% on LLaMA2 (13B) and 15.72% on LLaMA3 (8B) compared to manual CoT.
Large Language Models (LLMs) have recently achieved impressive results in complex reasoning tasks through Chain of Thought (CoT) prompting. However, most existing CoT methods rely on using the same prompts, whether manually designed or automatically generated, to handle the entire dataset. This one-size-fits-all approach may fail to meet the specific needs arising from the diversities within a single dataset. To solve this problem, we propose the Clustered Distance-Weighted Chain of Thought (CDW-CoT) method, which dynamically constructs prompts tailored to the characteristics of each data instance by integrating clustering and prompt optimization techniques. Our method employs clustering algorithms to categorize the dataset into distinct groups, from which a candidate pool of prompts is selected to reflect the inherent diversity within the dataset. For each cluster, CDW-CoT trains the optimal prompt probability distribution tailored to their specific characteristics. Finally, it dynamically constructs a unique prompt probability distribution for each test instance, based on its proximity to cluster centers, from which prompts are selected for reasoning. CDW-CoT consistently outperforms traditional CoT methods across six datasets, including commonsense, symbolic, and mathematical reasoning tasks. Specifically, when compared to manual CoT, CDW-CoT achieves an average accuracy improvement of 25.34% on LLaMA2 (13B) and 15.72% on LLaMA3 (8B).