Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcement
This addresses robustness issues in RAG systems for knowledge-intensive tasks, offering a method to reduce errors from misleading external context, though it is an incremental improvement on existing RAG techniques.
The paper tackles the problem of retrieval-augmented generation (RAG) being derailed by inaccurate or conflicting retrieved text, proposing Knowledgeable-R1, a reinforcement-learning framework that trains models to use parametric knowledge to resist such interference, resulting in a 23% improvement in counterfactual scenarios over SOTA baselines without degradation when context is accurate.
Retrieval-augmented generation (RAG) improves performance on knowledge-intensive tasks but can be derailed by wrong, irrelevant, or conflicting retrieved text, causing models to rely on inaccurate evidence and cascade errors. We propose Knowledgeable-R1, a reinforcement-learning framework that explicitly trains large language models to use parametric knowledge (PK) to resist contextual interference while still exploiting external context when it is reliably helpful. Knowledgeable-R1 introduces a joint sampling scheme that generates paired responses with and without retrieval, and learns both local advantages (within each decoding regime) and global advantages under the same input to quantify when to ignore misleading context versus adopt it. We employ an asymmetric advantage transformation that amplifies exploratory behaviors toward parametric knowledge. Experiments show that \method significantly improves robustness and reasoning accuracy in knowledge conflict scenarios and general RAG scenarios, outperforming SOTA baselines by 23% in counterfactual scenarios, and without degradation when the retrieved context is fully accurate.Our code are available at https://github.com/lcy80366872/knowledgeable-R1.