Software Engineering Agents for Embodied Controller Generation : A Study in Minigrid Environments
This work establishes controller generation for embodied tasks as a crucial evaluation domain for SWE-Agents, providing baseline results for future research in efficient reasoning systems, but it is incremental as it extends existing methods to a new application area.
The study tackled the problem of evaluating Software Engineering Agents (SWE-Agents) for generating controllers in embodied tasks, specifically in Minigrid environments, by adapting Mini-SWE-Agent to solve 20 diverse tasks and comparing performance across different information access conditions.
Software Engineering Agents (SWE-Agents) have proven effective for traditional software engineering tasks with accessible codebases, but their performance for embodied tasks requiring well-designed information discovery remains unexplored. We present the first extended evaluation of SWE-Agents on controller generation for embodied tasks, adapting Mini-SWE-Agent (MSWEA) to solve 20 diverse embodied tasks from the Minigrid environment. Our experiments compare agent performance across different information access conditions: with and without environment source code access, and with varying capabilities for interactive exploration. We quantify how different information access levels affect SWE-Agent performance for embodied tasks and analyze the relative importance of static code analysis versus dynamic exploration for task solving. This work establishes controller generation for embodied tasks as a crucial evaluation domain for SWE-Agents and provides baseline results for future research in efficient reasoning systems.