An Effective Framework to Help Large Language Models Handle Numeric-involved Long-context Tasks
This addresses a specific bottleneck for users of LLMs in numeric-involved long-context scenarios, offering a practical improvement.
The paper tackles the problem of large language models (LLMs) struggling with numerical calculations in long-context tasks, proposing a workflow that decomposes tasks into subtasks to improve accuracy and reduce API call costs, as demonstrated on two benchmarks.
Large Language Models (LLMs) have demonstrated remarkable capabilities in handling long texts and have almost perfect performance in traditional retrieval tasks. However, their performance significantly degrades when it comes to numerical calculations in the long-context. Numeric-involved long-context tasks typically cannot be addressed by current LLMs in normal settings due to their inherent limitations in simultaneously handling complex and massive information. Some CoT like prompting methods can improve accuracy but demands massive output tokens, which is costly and slow. To address this issue, we propose a workflow, which decompose a numeric-involved long-context task into 4 low-level subtasks: judging, extracting and processing with code and conclusion. The former 2 subtasks is relatively simple, which allows us to use smaller models for efficiently processing long context. When numerical calculations are required, we use code generated by LLMs to avoid the disadvantage of LLM not being good at calculations. The results in 2 numeric-involved long-context benchmarks demonstrate our workflow can not only improve accuracy, but also significantly reduce the cost of API calls.