Is Implicit Knowledge Enough for LLMs? A RAG Approach for Tree-based Structures
This addresses a domain-specific problem for developers or researchers working with hierarchical data in LLMs, offering an incremental improvement in efficiency for RAG systems.
The paper tackled the problem of representing retrieved knowledge for generating responses on hierarchical data structures like trees in Retrieval-Augmented Generation (RAG), proposing a bottom-up method to linearize knowledge with aggregated summaries. The result showed comparable response quality but generated over 68% fewer documents in the retriever, indicating significant efficiency gains.
Large Language Models (LLMs) are adept at generating responses based on information within their context. While this ability is useful for interacting with structured data like code files, another popular method, Retrieval-Augmented Generation (RAG), retrieves relevant documents to augment the model's in-context learning. However, it is not well-explored how to best represent this retrieved knowledge for generating responses on structured data, particularly hierarchical structures like trees. In this work, we propose a novel bottom-up method to linearize knowledge from tree-like structures (like a GitHub repository) by generating implicit, aggregated summaries at each hierarchical level. This approach enables the knowledge to be stored in a knowledge base and used directly with RAG. We then compare our method to using RAG on raw, unstructured code, evaluating the accuracy and quality of the generated responses. Our results show that while response quality is comparable across both methods, our approach generates over 68% fewer documents in the retriever, a significant gain in efficiency. This finding suggests that leveraging implicit, linearized knowledge may be a highly effective and scalable strategy for handling complex, hierarchical data structures.