SAKR: Enhancing Retrieval-Augmented Generation via Streaming Algorithm and K-Means Clustering
This addresses memory and update inefficiencies in RAG systems for information retrieval with LLMs, but it is incremental as it builds on existing RAG methods.
The paper tackled the high memory consumption and inability to update indices in real-time for retrieval-augmented generation (RAG) systems by integrating a streaming algorithm and k-means clustering, resulting in improved accuracy and reduced memory usage, especially with large-scale streaming data.
Retrieval-augmented generation (RAG) has achieved significant success in information retrieval to assist large language models LLMs because it builds an external knowledge database. However, it also has many problems, it consumes a lot of memory because of the enormous database, and it cannot update the established index database in time when confronted with massive streaming data. To reduce the memory required for building the database and maintain accuracy simultaneously, we proposed a new approach integrating a streaming algorithm with k-means clustering into RAG. Our approach applied a streaming algorithm to update the index dynamically and reduce memory consumption. Additionally, the k-means algorithm clusters highly similar documents, and the query time would be shortened. We conducted comparative experiments on four methods, and the results indicated that RAG with streaming algorithm and k-means clusters outperforms traditional RAG in accuracy and memory, particularly when dealing with large-scale streaming data.