SKETCH: Structured Knowledge Enhanced Text Comprehension for Holistic Retrieval
This addresses the challenge of enhancing retrieval accuracy and context integrity in RAG systems for applications relying on large datasets, though it appears incremental as it builds on existing RAG frameworks.
The paper tackles the problem of inefficient retrieval and context comprehension in Retrieval-Augmented Generation (RAG) systems by introducing SKETCH, which integrates semantic text retrieval with knowledge graphs, resulting in improved performance such as an answer relevancy of 0.94 and context precision of 0.99 on the Italian Cuisine dataset.
Retrieval-Augmented Generation (RAG) systems have become pivotal in leveraging vast corpora to generate informed and contextually relevant responses, notably reducing hallucinations in Large Language Models. Despite significant advancements, these systems struggle to efficiently process and retrieve information from large datasets while maintaining a comprehensive understanding of the context. This paper introduces SKETCH, a novel methodology that enhances the RAG retrieval process by integrating semantic text retrieval with knowledge graphs, thereby merging structured and unstructured data for a more holistic comprehension. SKETCH, demonstrates substantial improvements in retrieval performance and maintains superior context integrity compared to traditional methods. Evaluated across four diverse datasets: QuALITY, QASPER, NarrativeQA, and Italian Cuisine-SKETCH consistently outperforms baseline approaches on key RAGAS metrics such as answer_relevancy, faithfulness, context_precision and context_recall. Notably, on the Italian Cuisine dataset, SKETCH achieved an answer relevancy of 0.94 and a context precision of 0.99, representing the highest performance across all evaluated metrics. These results highlight SKETCH's capability in delivering more accurate and contextually relevant responses, setting new benchmarks for future retrieval systems.