A Graph-based RAG for Energy Efficiency Question Answering
This work addresses energy efficiency information retrieval for users in the energy domain, but it is incremental as it applies existing RAG and graph methods to a new dataset.
The paper tackled energy efficiency question answering by developing a graph-based RAG system that extracts knowledge graphs from documents and uses LLMs for reasoning, achieving 75.2% accuracy on a validation dataset with up to 81.0% on general questions and a 4.4% accuracy loss in multilingual settings.
In this work, we investigate the use of Large Language Models (LLMs) within a graph-based Retrieval Augmented Generation (RAG) architecture for Energy Efficiency (EE) Question Answering. First, the system automatically extracts a Knowledge Graph (KG) from guidance and regulatory documents in the energy field. Then, the generated graph is navigated and reasoned upon to provide users with accurate answers in multiple languages. We implement a human-based validation using the RAGAs framework properties, a validation dataset comprising 101 question-answer pairs, and domain experts. Results confirm the potential of this architecture and identify its strengths and weaknesses. Validation results show how the system correctly answers in about three out of four of the cases (75.2 +- 2.7%), with higher results on questions related to more general EE answers (up to 81.0 +- 4.1%), and featuring promising multilingual abilities (4.4% accuracy loss due to translation).