KGRAG-Ex: Explainable Retrieval-Augmented Generation with Knowledge Graph-based Perturbations
This addresses the problem of opaque reasoning in RAG systems for users needing transparent AI responses, though it appears incremental as it builds on existing KG-based methods.
The paper tackles the challenge of explainability in Retrieval-Augmented Generation (RAG) by introducing KGRAG-Ex, a system that uses knowledge graphs to improve factual grounding and interpretability, with experiments analyzing sensitivity to perturbations and relationships between graph components and explanations.
Retrieval-Augmented Generation (RAG) enhances language models by grounding responses in external information, yet explainability remains a critical challenge, particularly when retrieval relies on unstructured text. Knowledge graphs (KGs) offer a solution by introducing structured, semantically rich representations of entities and their relationships, enabling transparent retrieval paths and interpretable reasoning. In this work, we present KGRAG-Ex, a RAG system that improves both factual grounding and explainability by leveraging a domain-specific KG constructed via prompt-based information extraction. Given a user query, KGRAG-Ex identifies relevant entities and semantic paths in the graph, which are then transformed into pseudo-paragraphs: natural language representations of graph substructures that guide corpus retrieval. To improve interpretability and support reasoning transparency, we incorporate perturbation-based explanation methods that assess the influence of specific KG-derived components on the generated answers. We conduct a series of experiments to analyze the sensitivity of the system to different perturbation methods, the relationship between graph component importance and their structural positions, the influence of semantic node types, and how graph metrics correspond to the influence of components within the explanations process.