Micro-Act: Mitigating Knowledge Conflict in LLM-based RAG via Actionable Self-Reasoning
This addresses a critical bottleneck in RAG systems for question-answering tasks, offering a robust solution that works on both conflict and non-conflict scenarios, though it is incremental as it builds on existing mitigation approaches.
The paper tackles the problem of knowledge conflict in Retrieval-Augmented Generation (RAG) systems, where external knowledge contradicts LLM parametric knowledge, and proposes Micro-Act, a framework that improves QA accuracy by 5-15% over state-of-the-art baselines across five datasets and three conflict types.
Retrieval-Augmented Generation (RAG) systems commonly suffer from Knowledge Conflicts, where retrieved external knowledge contradicts the inherent, parametric knowledge of large language models (LLMs). It adversely affects performance on downstream tasks such as question answering (QA). Existing approaches often attempt to mitigate conflicts by directly comparing two knowledge sources in a side-by-side manner, but this can overwhelm LLMs with extraneous or lengthy contexts, ultimately hindering their ability to identify and mitigate inconsistencies. To address this issue, we propose Micro-Act a framework with a hierarchical action space that automatically perceives context complexity and adaptively decomposes each knowledge source into a sequence of fine-grained comparisons. These comparisons are represented as actionable steps, enabling reasoning beyond the superficial context. Through extensive experiments on five benchmark datasets, Micro-Act consistently achieves significant increase in QA accuracy over state-of-the-art baselines across all 5 datasets and 3 conflict types, especially in temporal and semantic types where all baselines fail significantly. More importantly, Micro-Act exhibits robust performance on non-conflict questions simultaneously, highlighting its practical value in real-world RAG applications.