Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration
This addresses the limitation of existing LLM-based agents that focus only on local code, improving automated issue resolution for software developers, though it is incremental as it builds on prior agent methods.
The paper tackles the problem of automated software issue resolution by introducing Alibaba LingmaAgent, a method that comprehensively explores entire software repositories using a knowledge graph and Monte Carlo tree search, achieving an 18.5% relative improvement on the SWE-bench Lite benchmark and resolving 16.9% of in-house issues automatically.
This paper presents Alibaba LingmaAgent, a novel Automated Software Engineering method designed to comprehensively understand and utilize whole software repositories for issue resolution. Deployed in TONGYI Lingma, an IDE-based coding assistant developed by Alibaba Cloud, LingmaAgent addresses the limitations of existing LLM-based agents that primarily focus on local code information. Our approach introduces a top-down method to condense critical repository information into a knowledge graph, reducing complexity, and employs a Monte Carlo tree search based strategy enabling agents to explore and understand entire repositories. We guide agents to summarize, analyze, and plan using repository-level knowledge, allowing them to dynamically acquire information and generate patches for real-world GitHub issues. In extensive experiments, LingmaAgent demonstrated significant improvements, achieving an 18.5\% relative improvement on the SWE-bench Lite benchmark compared to SWE-agent. In production deployment and evaluation at Alibaba Cloud, LingmaAgent automatically resolved 16.9\% of in-house issues faced by development engineers, and solved 43.3\% of problems after manual intervention. Additionally, we have open-sourced a Python prototype of LingmaAgent for reference by other industrial developers https://github.com/RepoUnderstander/RepoUnderstander. In fact, LingmaAgent has been used as a developed reference by many subsequently agents.