Keyword search is all you need: Achieving RAG-Level Performance without vector databases using agentic tool use
This offers a simpler, cost-effective alternative for question-answering in scenarios with frequently updated knowledge bases, though it is incremental as it builds on existing agentic-RAG and tool-augmented LLM architectures.
The study tackled the problem of Retrieval-Augmented Generation (RAG) systems being complex and costly by comparing them to agentic keyword search tools, finding that keyword search achieves over 90% of RAG's performance metrics without vector databases.
While Retrieval-Augmented Generation (RAG) has proven effective for generating accurate, context-based responses based on existing knowledge bases, it presents several challenges including retrieval quality dependencies, integration complexity and cost. Recent advances in agentic-RAG and tool-augmented LLM architectures have introduced alternative approaches to information retrieval and processing. We question how much additional value vector databases and semantic search bring to RAG over simple, agentic keyword search in documents for question-answering. In this study, we conducted a systematic comparison between RAG-based systems and tool-augmented LLM agents, specifically evaluating their retrieval mechanisms and response quality when the agent only has access to basic keyword search tools. Our empirical analysis demonstrates that tool-based keyword search implementations within an agentic framework can attain over $90\%$ of the performance metrics compared to traditional RAG systems without using a standing vector database. Our approach is simple to implement, cost effective, and is particularly useful in scenarios requiring frequent updates to knowledge bases.