The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models
This addresses efficiency and cost issues for users of large language models, but it is incremental as it builds on existing Chain-of-Thought methods.
The paper tackled the problem of reducing response length in large language models by introducing Concise Chain-of-Thought (CCoT) prompting, which reduced average response length by 48.70% with negligible impact on accuracy, except for a 27.69% performance penalty on math problems in GPT-3.5, leading to a 22.67% cost reduction.
In this paper, we introduce Concise Chain-of-Thought (CCoT) prompting. We compared standard CoT and CCoT prompts to see how conciseness impacts response length and correct-answer accuracy. We evaluated this using GPT-3.5 and GPT-4 with a multiple-choice question-and-answer (MCQA) benchmark. CCoT reduced average response length by 48.70% for both GPT-3.5 and GPT-4 while having a negligible impact on problem-solving performance. However, on math problems, GPT-3.5 with CCoT incurs a performance penalty of 27.69%. Overall, CCoT leads to an average per-token cost reduction of 22.67%. All code, data, and supplemental materials are available on GitHub at https://github.com/matthewrenze/jhu-concise-cot