Explainable Knowledge Graph Retrieval-Augmented Generation (KG-RAG) with KG-SMILE
This addresses the need for greater transparency and trust in machine learning technologies, especially for sensitive applications, though it appears incremental as it builds on existing RAG and interpretability methods.
The paper tackles the problem of hallucinations and lack of transparency in Retrieval-Augmented Generation (RAG) systems, particularly in sensitive domains like healthcare, by developing KG-SMILE, a method-agnostic framework that provides token and component-level interpretability, resulting in stable and human-aligned explanations that balance model effectiveness with interpretability.
Generative AI, such as Large Language Models (LLMs), has achieved impressive progress but still produces hallucinations and unverifiable claims, limiting reliability in sensitive domains. Retrieval-Augmented Generation (RAG) improves accuracy by grounding outputs in external knowledge, especially in domains like healthcare, where precision is vital. However, RAG remains opaque and essentially a black box, heavily dependent on data quality. We developed a method-agnostic, perturbation-based framework that provides token and component-level interoperability for Graph RAG using SMILE and named it as Knowledge-Graph (KG)-SMILE. By applying controlled perturbations, computing similarities, and training weighted linear surrogates, KG-SMILE identifies the graph entities and relations most influential to generated outputs, thereby making RAG more transparent. We evaluate KG-SMILE using comprehensive attribution metrics, including fidelity, faithfulness, consistency, stability, and accuracy. Our findings show that KG-SMILE produces stable, human-aligned explanations, demonstrating its capacity to balance model effectiveness with interpretability and thereby fostering greater transparency and trust in machine learning technologies.