ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation
This addresses persistent challenges in RAG systems for enhancing LLMs with external knowledge, though it appears incremental as it builds on existing RAG methods.
The paper tackles retrieval inefficiency and irrelevant information filtering in Retrieval-Augmented Generation (RAG) systems by introducing ParetoRAG, an unsupervised framework that uses sentence-level refinement based on the Pareto principle, achieving improvements in retrieval precision and generation quality without extra training or resources.
While Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by incorporating external knowledge, they still face persistent challenges in retrieval inefficiency and the inability of LLMs to filter out irrelevant information. We present ParetoRAG, an unsupervised framework that optimizes RAG systems through sentence-level refinement guided by the Pareto principle. By decomposing paragraphs into sentences and dynamically re-weighting core content while preserving contextual coherence, ParetoRAG achieves dual improvements in both retrieval precision and generation quality without requiring additional training or API resources. This framework has been empirically validated across various datasets, LLMs, and retrievers.