Conflict-Aware Soft Prompting for Retrieval-Augmented Generation
This addresses reliability issues in RAG systems for users needing trustworthy AI responses, though it is incremental as it builds on existing RAG methods.
The paper tackled the problem of context-memory conflict in retrieval-augmented generation, where retrieved external knowledge contradicts the model's internal knowledge, and introduced CARE, which improved performance by an average of 5.0% on QA and fact-checking benchmarks.
Retrieval-augmented generation (RAG) enhances the capabilities of large language models (LLMs) by incorporating external knowledge into their input prompts. However, when the retrieved context contradicts the LLM's parametric knowledge, it often fails to resolve the conflict between incorrect external context and correct parametric knowledge, known as context-memory conflict. To tackle this problem, we introduce Conflict-Aware REtrieval-Augmented Generation (CARE), consisting of a context assessor and a base LLM. The context assessor encodes compact memory token embeddings from raw context tokens. Through grounded/adversarial soft prompting, the context assessor is trained to discern unreliable context and capture a guidance signal that directs reasoning toward the more reliable knowledge source. Extensive experiments show that CARE effectively mitigates context-memory conflicts, leading to an average performance gain of 5.0\% on QA and fact-checking benchmarks, establishing a promising direction for trustworthy and adaptive RAG systems.