Improving Performance of Commercially Available AI Products in a Multi-Agent Configuration
This addresses the gap in real-world applications of multi-agent systems for software development, though it is incremental as it builds on existing commercial tools.
The study tackled the problem of integrating commercially available AI tools in a multi-agent system to enhance software development, showing that sharing business requirements from Crowdbotics PRD AI improved GitHub Copilot's code suggestions by 13.8% and developer task success rate by 24.5%.
In recent years, with the rapid advancement of large language models (LLMs), multi-agent systems have become increasingly more capable of practical application. At the same time, the software development industry has had a number of new AI-powered tools developed that improve the software development lifecycle (SDLC). Academically, much attention has been paid to the role of multi-agent systems to the SDLC. And, while single-agent systems have frequently been examined in real-world applications, we have seen comparatively few real-world examples of publicly available commercial tools working together in a multi-agent system with measurable improvements. In this experiment we test context sharing between Crowdbotics PRD AI, a tool for generating software requirements using AI, and GitHub Copilot, an AI pair-programming tool. By sharing business requirements from PRD AI, we improve the code suggestion capabilities of GitHub Copilot by 13.8% and developer task success rate by 24.5% -- demonstrating a real-world example of commercially-available AI systems working together with improved outcomes.