TOM-SWE: User Mental Modeling For Software Engineering Agents
This addresses the challenge of user intent inference for software engineering agents, offering a novel approach with practical gains, though it is incremental in enhancing existing agent architectures.
The paper tackles the problem of coding agents struggling to infer user intent in underspecified or context-dependent instructions by introducing ToM-SWE, a dual-agent architecture with a theory-of-mind partner for user mental modeling. It improves task success rates, achieving 59.7% vs. 18.1% for a state-of-the-art agent on a stateful benchmark, and is found useful 86% of the time in a study with professional developers.
Recent advances in coding agents have made them capable of planning, editing, running, and testing complex code bases. Despite their growing ability in coding tasks, these systems still struggle to infer and track user intent, especially when instructions are underspecified or context-dependent. To bridge this gap, we introduce ToM-SWE, a dual-agent architecture that pairs a primary software-engineering (SWE) agent with a lightweight theory-of-mind (ToM) partner agent dedicated to modeling the user's mental state. The ToM agent infers user goals, constraints, and preferences from instructions and interaction history, maintains a \textbf{persistent memory} of the user, and provides user-related suggestions to the SWE agent. In two software engineering benchmarks (ambiguous SWE-bench and stateful SWE-bench), ToM-SWE improves task success rates and user satisfaction. Notably, on the stateful SWE benchmark, a newly introduced evaluation that provides agents with a user simulator along with previous interaction histories, ToM-SWE achieves a substantially higher task success rate of 59.7\% compared to 18.1\% for OpenHands, a state-of-the-art SWE agent. Furthermore, in a three-week study with professional developers using ToM-SWE in their daily work, participants found it useful 86\% of the time, underscoring the value of stateful user modeling for practical coding agents.