XGraphRAG: Interactive Visual Analysis for Graph-based Retrieval-Augmented Generation
This addresses interpretability and accessibility issues for developers using GraphRAG, but it is incremental as it builds on existing GraphRAG methods.
The paper tackles the challenge of analyzing GraphRAG's effectiveness due to its complex pipeline and high LLM usage, proposing XGraphRAG, a visual analysis framework that helps developers identify critical recalls and trace them, with evaluation showing effectiveness and usability.
Graph-based Retrieval-Augmented Generation (RAG) has shown great capability in enhancing Large Language Model (LLM)'s answer with an external knowledge base. Compared to traditional RAG, it introduces a graph as an intermediate representation to capture better structured relational knowledge in the corpus, elevating the precision and comprehensiveness of generation results. However, developers usually face challenges in analyzing the effectiveness of GraphRAG on their dataset due to GraphRAG's complex information processing pipeline and the overwhelming amount of LLM invocations involved during graph construction and query, which limits GraphRAG interpretability and accessibility. This research proposes a visual analysis framework that helps RAG developers identify critical recalls of GraphRAG and trace these recalls through the GraphRAG pipeline. Based on this framework, we develop XGraphRAG, a prototype system incorporating a set of interactive visualizations to facilitate users' analysis process, boosting failure cases collection and improvement opportunities identification. Our evaluation demonstrates the effectiveness and usability of our approach. Our work is open-sourced and available at https://github.com/Gk0Wk/XGraphRAG.