CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology
For practitioners and researchers in AI-assisted software development, this work provides empirical insights into real-world applicability of LLM-based multi-agent systems, but the findings are incremental and qualitative.
The paper presents CodePori, a multi-agent system for autonomous software development, and evaluates it through participant feedback, revealing strengths and challenges such as memory limitations and hallucinations that are not captured by standard benchmarks.
Context: LLM-based multi-agent systems enable automation and decision support in software development, yet existing studies rely on benchmark datasets offering only binary pass-or-fail results, limiting insight into real-world applicability. Objective: This study empirically investigates the potential and limitations of LLM-based agents in autonomous software development tasks. Method: A two-phase approach was employed: developing a multi-agent system, CodePori, for automated code generation, and conducting participant-based evaluation to assess practical performance. Results: Participant feedback reveals key strengths, challenges, and areas for improvement in LLM-based multi-agent systems, highlighting aspects missed by standard code-generation benchmarks. Conclusions: While LLM-based multi-agent systems show potential for large-scale software development, successful integration requires addressing challenges such as memory limitations, hallucinations, and code smells, alongside a practitioner-centric perspective.