Cache Mechanism for Agent RAG Systems
This addresses efficiency and effectiveness issues for developers and users of RAG-powered LLM agents, representing a novel method for a known bottleneck rather than an incremental improvement.
The paper tackles the problem of inefficient cache management in RAG-based LLM agents by introducing ARC, a novel caching framework that reduces storage to 0.015% of the original corpus, achieves up to 79.8% has-answer rate, and cuts average retrieval latency by 80%.
Recent advances in Large Language Model (LLM)-based agents have been propelled by Retrieval-Augmented Generation (RAG), which grants the models access to vast external knowledge bases. Despite RAG's success in improving agent performance, agent-level cache management, particularly constructing, maintaining, and updating a compact, relevant corpus dynamically tailored to each agent's need, remains underexplored. Therefore, we introduce ARC (Agent RAG Cache Mechanism), a novel, annotation-free caching framework that dynamically manages small, high-value corpora for each agent. By synthesizing historical query distribution patterns with the intrinsic geometry of cached items in the embedding space, ARC automatically maintains a high-relevance cache. With comprehensive experiments on three retrieval datasets, our experimental results demonstrate that ARC reduces storage requirements to 0.015% of the original corpus while offering up to 79.8% has-answer rate and reducing average retrieval latency by 80%. Our results demonstrate that ARC can drastically enhance efficiency and effectiveness in RAG-powered LLM agents.