AssistRAG: Boosting the Potential of Large Language Models with an Intelligent Information Assistant
This addresses the issue of factual inaccuracies in LLMs for users relying on complex reasoning tasks, representing an incremental advancement over existing RAG methods.
The paper tackles the problem of hallucination in Large Language Models by proposing AssistRAG, an intelligent information assistant that integrates memory, knowledge, and planning to enhance retrieval-augmented generation, resulting in significant performance improvements over benchmarks, particularly for less advanced models.
The emergence of Large Language Models (LLMs) has significantly advanced natural language processing, but these models often generate factually incorrect information, known as "hallucination". Initial retrieval-augmented generation (RAG) methods like the "Retrieve-Read" framework was inadequate for complex reasoning tasks. Subsequent prompt-based RAG strategies and Supervised Fine-Tuning (SFT) methods improved performance but required frequent retraining and risked altering foundational LLM capabilities. To cope with these challenges, we propose Assistant-based Retrieval-Augmented Generation (AssistRAG), integrating an intelligent information assistant within LLMs. This assistant manages memory and knowledge through tool usage, action execution, memory building, and plan specification. Using a two-phase training approach, Curriculum Assistant Learning and Reinforced Preference Optimization. AssistRAG enhances information retrieval and decision-making. Experiments show AssistRAG significantly outperforms benchmarks, especially benefiting less advanced LLMs, by providing superior reasoning capabilities and accurate responses.