From Search to Reasoning: A Five-Level RAG Capability Framework for Enterprise Data
This provides a problem-oriented framework for enterprise users to improve question-answering systems, though it is incremental as it builds on existing RAG techniques.
The paper tackles the limitations of traditional Retrieval-Augmented Generation (RAG) in handling complex questions and non-text data by proposing a five-level classification framework (L1-L5) to categorize systems based on data modalities and task complexity, and introduces benchmarks to evaluate four state-of-the-art platforms, showing that multi-space retrieval and dynamic orchestration enable L1-L4 capabilities.
Retrieval-Augmented Generation (RAG) has emerged as the standard paradigm for answering questions on enterprise data. Traditionally, RAG has centered on text-based semantic search and re-ranking. However, this approach falls short when dealing with questions beyond data summarization or non-text data. This has led to various attempts to supplement RAG to bridge the gap between RAG, the implementation paradigm, and the question answering problem that enterprise users expect it to solve. Given that contemporary RAG is a collection of techniques rather than a defined implementation, discussion of RAG and related question-answering systems benefits from a problem-oriented understanding. We propose a new classification framework (L1-L5) to categorize systems based on data modalities and task complexity of the underlying question answering problems: L1 (Surface Knowledge of Unstructured Data) through L4 (Reflective and Reasoned Knowledge) and the aspirational L5 (General Intelligence). We also introduce benchmarks aligned with these levels and evaluate four state-of-the-art platforms: LangChain, Azure AI Search, OpenAI, and Corvic AI. Our experiments highlight the value of multi-space retrieval and dynamic orchestration for enabling L1-L4 capabilities. We empirically validate our findings using diverse datasets indicative of enterprise use cases.