Scaling Environments for LLM Agents in the Era of Learning from Interaction: A Survey
This addresses the need for more dynamic and realistic training environments for LLM agents to improve adaptive behavior and long-term decision-making, but it is incremental as it surveys existing advances rather than introducing new methods.
The paper tackles the problem of insufficient static datasets for training LLM-based agents by proposing the Generation-Execution-Feedback (GEF) loop, where environments generate tasks and provide feedback to enable learning from interaction, and it reviews methods for scaling environments to enhance complexity and realism.
LLM-based agents can autonomously accomplish complex tasks across various domains. However, to further cultivate capabilities such as adaptive behavior and long-term decision-making, training on static datasets built from human-level knowledge is insufficient. These datasets are costly to construct and lack both dynamism and realism. A growing consensus is that agents should instead interact directly with environments and learn from experience through reinforcement learning. We formalize this iterative process as the Generation-Execution-Feedback (GEF) loop, where environments generate tasks to challenge agents, return observations in response to agents' actions during task execution, and provide evaluative feedback on rollouts for subsequent learning. Under this paradigm, environments function as indispensable producers of experiential data, highlighting the need to scale them toward greater complexity, realism, and interactivity. In this survey, we systematically review representative methods for environment scaling from a pioneering environment-centric perspective and organize them along the stages of the GEF loop, namely task generation, task execution, and feedback. We further analyze benchmarks, implementation strategies, and applications, consolidating fragmented advances and outlining future research directions for agent intelligence.