Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning
This work addresses cost efficiency for users of large language models in multi-step reasoning, offering an incremental improvement over existing adaptive methods.
The paper tackles the high computational cost of self-consistency in chain-of-thought reasoning by proposing Difficulty-Adaptive Self-Consistency (DSC), which adaptively allocates inference resources based on question difficulty, reducing costs by a significant margin while maintaining comparable performance on six benchmarks across arithmetic, commonsense, and symbolic reasoning tasks.
Self-consistency (SC), a widely used decoding strategy for chain-of-thought reasoning, shows significant gains across various multi-step reasoning tasks but comes with a high cost due to multiple sampling with the preset size. Its variants, Adaptive self-consistency (ASC) and Early-stopping self-consistency (ESC), dynamically adjust the number of samples based on the posterior distribution of a set of pre-samples, reducing the cost of SC with minimal impact on performance. Both methods, however, do not exploit the prior information about question difficulty. It often results in unnecessary repeated sampling for easy questions that could be accurately answered with just one attempt, wasting resources. To tackle this problem, we propose Difficulty-Adaptive Self-Consistency (DSC), which leverages the difficulty information of batch queries from both prior and posterior perspectives to adaptively allocate inference resources, further reducing the overall cost of SC. To demonstrate the effectiveness of DSC, we conduct extensive experiments on three popular categories of reasoning tasks: arithmetic, commonsense and symbolic reasoning on six benchmarks. The empirical results show that DSC consistently surpasses the strong baseline ASC and ESC in terms of costs by a significant margin, while attaining comparable performances.