ChunkRAG: Novel LLM-Chunk Filtering Method for RAG Systems
This work addresses the reliability of RAG systems for applications like fact-checking and multi-hop reasoning, representing an incremental improvement over existing methods.
The paper tackles the problem of inaccurate responses in Retrieval-Augmented Generation (RAG) systems by proposing ChunkRAG, a framework that filters retrieved information at the chunk level using LLM-based relevance scoring, which reduces hallucinations and improves factual accuracy.
Retrieval-Augmented Generation (RAG) systems using large language models (LLMs) often generate inaccurate responses due to the retrieval of irrelevant or loosely related information. Existing methods, which operate at the document level, fail to effectively filter out such content. We propose LLM-driven chunk filtering, ChunkRAG, a framework that enhances RAG systems by evaluating and filtering retrieved information at the chunk level. Our approach employs semantic chunking to divide documents into coherent sections and utilizes LLM-based relevance scoring to assess each chunk's alignment with the user's query. By filtering out less pertinent chunks before the generation phase, we significantly reduce hallucinations and improve factual accuracy. Experiments show that our method outperforms existing RAG models, achieving higher accuracy on tasks requiring precise information retrieval. This advancement enhances the reliability of RAG systems, making them particularly beneficial for applications like fact-checking and multi-hop reasoning.