EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systems
This addresses the limitation of AI agents in adapting to novel environments during deployment, which is crucial for practical applications like autonomous systems, though it appears incremental as it builds on existing evolutionary and agentic concepts.
The paper tackles the problem of AI agents' inability to learn complex skills at test time by introducing the Jericho Test-Time Learning (J-TTL) benchmark and proposing EvoTest, an evolutionary framework that evolves agent configurations without fine-tuning, resulting in consistent performance gains and the only method capable of winning two games where baselines failed.
A fundamental limitation of current AI agents is their inability to learn complex skills on the fly at test time, often behaving like "clever but clueless interns" in novel environments. This severely limits their practical utility. To systematically measure and drive progress on this challenge, we first introduce the Jericho Test-Time Learning (J-TTL) benchmark. J-TTL is a new evaluation setup where an agent must play the same game for several consecutive episodes, attempting to improve its performance from one episode to the next. On J-TTL, we find that existing adaptation methods like reflection, memory, or reinforcement learning struggle. To address the challenges posed by our benchmark, we present EvoTest, an evolutionary test-time learning framework that improves an agent without any fine-tuning or gradients-by evolving the entire agentic system after every episode. EvoTest has two roles: the Actor Agent, which plays the game, and the Evolver Agent, which analyzes the episode transcript to propose a revised configuration for the next run. This configuration rewrites the prompt, updates memory by logging effective state-action choices, tunes hyperparameters, and learns the tool-use routines. On our J-TTL benchmark, EvoTest consistently increases performance, outperforming not only reflection and memory-only baselines but also more complex online fine-tuning methods. Notably, our method is the only one capable of winning two games (Detective and Library), while all baselines fail to win any.