Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filtering
This addresses the challenge of processing long documents or extended conversations for LLM users, but it is incremental as it builds on existing context limitation issues.
The paper tackled the problem of fixed context length in large language models (LLMs) by proposing Selective Context, a method that uses self-information to filter less informative content, enhancing context efficiency; it demonstrated effectiveness on summarization and question answering tasks across various data sources.
Large language models (LLMs) have received significant attention by achieving remarkable performance across various tasks. However, their fixed context length poses challenges when processing long documents or maintaining extended conversations. This paper proposes a method called \textit{Selective Context} that employs self-information to filter out less informative content, thereby enhancing the efficiency of the fixed context length. We demonstrate the effectiveness of our approach on tasks of summarisation and question answering across different data sources, including academic papers, news articles, and conversation transcripts.