A + B: A General Generator-Reader Framework for Optimizing LLMs to Unleash Synergy Potential
This work addresses performance limitations in LLMs for knowledge-intensive tasks, offering an incremental improvement with potential domain-specific applications.
The paper tackles the bottleneck in Retrieval-Augmented Generation (RAG) by proposing a general 'A + B' framework that combines different foundation models as generator and reader, consistently outperforming single models in complex scenarios and extending it to integrate external knowledge through continuous learning.
Retrieval-Augmented Generation (RAG) is an effective solution to supplement necessary knowledge to large language models (LLMs). Targeting its bottleneck of retriever performance, "generate-then-read" pipeline is proposed to replace the retrieval stage with generation from the LLM itself. Although promising, this research direction is underexplored and still cannot work in the scenario when source knowledge is given. In this paper, we formalize a general "A + B" framework with varying combinations of foundation models and types for systematic investigation. We explore the efficacy of the base and chat versions of LLMs and found their different functionalities suitable for generator A and reader B, respectively. Their combinations consistently outperform single models, especially in complex scenarios. Furthermore, we extend the application of the "A + B" framework to scenarios involving source documents through continuous learning, enabling the direct integration of external knowledge into LLMs. This approach not only facilitates effective acquisition of new knowledge but also addresses the challenges of safety and helpfulness post-adaptation. The paper underscores the versatility of the "A + B" framework, demonstrating its potential to enhance the practical application of LLMs across various domains.