DeepRefine: Agent-Compiled Knowledge Refinement via Reinforcement Learning
For LLM agents relying on persistent external knowledge, DeepRefine addresses the critical bottleneck of knowledge base defects (incompleteness, incorrectness, redundancy) that degrade retrieval and downstream performance.
DeepRefine improves the quality of agent-compiled knowledge bases by using reinforcement learning to train an LLM-based reasoning model that performs multi-turn interactions, abductive diagnosis, and targeted refinement, achieving consistent downstream gains over strong baselines.
Agent-compiled knowledge bases provide persistent external knowledge for large language model (LLM) agents in open-ended, knowledge-intensive downstream tasks. Yet their quality is systematically limited by \emph{incompleteness}, \emph{incorrectness}, and \emph{redundancy}, manifested as missing evidence or cross-document links, low-confidence or imprecise claims, and ambiguous or coreference resolution issues. Such defects compound under iterative use, degrading retrieval fidelity and downstream task performance. We present \textbf{DeepRefine}, a general LLM-based reasoning model for \emph{agent-compiled knowledge refinement} that improves the quality of any pre-constructed knowledge bases with user queries to make it more suitable for the downstream tasks. DeepRefine performs multi-turn interactions with the knowledge base and conducts abductive diagnosis over interaction history, localizes likely defects, and executes targeted refinement actions for incremental knowledge base updates. To optimize refinement policies of DeepRefine without gold references, we introduce a Gain-Beyond-Draft (GBD) reward and train the reasoning process end-to-end via reinforcement learning. Extensive experiments demonstrate consistent downstream gains over strong baselines.