EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle
This work addresses a fundamental limitation in LLM agents for AI researchers and developers, offering a blueprint for more autonomous and continuously improving systems, though it appears incremental as it builds on existing agent frameworks.
The paper tackles the problem of LLM agents lacking systematic learning from their own experiences by introducing EvolveR, a framework that enables self-improvement through an experience-driven lifecycle, achieving superior performance on complex multi-hop question-answering benchmarks over strong baselines.
Current Large Language Model (LLM) agents show strong performance in tool use, but lack the crucial capability to systematically learn from their own experiences. While existing frameworks mainly focus on mitigating external knowledge gaps, they fail to address a more fundamental limitation: the inability to iteratively refine problem-solving strategies. In this work, we introduce EvolveR, a framework designed to enable agent to self-improve through a complete, closed-loop experience lifecycle. This lifecycle comprises two key stages: (1) Offline Self-Distillation, where the agent's interaction trajectories are synthesized into a structured repository of abstract, reusable strategic principles; (2) Online Interaction, where the agent interacts with tasks and actively retrieves distilled principles to guide its decision-making, accumulating a diverse set of behavioral trajectories. This loop employs a policy reinforcement mechanism to iteratively update the agent based on its performance. We demonstrate the effectiveness of EvolveR on complex multi-hop question-answering benchmarks, where it achieves superior performance over strong agentic baselines. Our work presents a comprehensive blueprint for agents that learn not only from external data but also from the consequences of their own actions, paving the way for more autonomous and continuously improving systems. Code is available at https://github.com/Edaizi/EvolveR.