AgentSZZ: Teaching the LLM Agent to Play Detective with Bug-Inducing Commits
This addresses a key bottleneck in software engineering tasks like defect prediction, offering substantial improvements over existing methods, though it builds incrementally on prior LLM-based SZZ approaches.
The paper tackles the problem of identifying bug-inducing commits in software engineering, where existing SZZ algorithms have limited performance due to reliance on git blame and fixed pipelines. The proposed AgentSZZ framework uses LLM-driven agents with task-specific tools and a ReAct-style loop, achieving F1-score gains of up to 27.2% over prior LLM-based approaches and recall gains of up to 300% for cross-file commits.
The SZZ algorithm is the dominant technique for identifying bug-inducing commits and underpins many software engineering tasks, such as defect prediction and vulnerability analysis. Despite numerous variants, including recent LLM-based approaches, performance remains limited on developer-annotated datasets (e.g., recall of 0.552 on the Linux kernel). A key limitation is the reliance on git blame, which traces line-level changes within the same file, failing in common scenarios such as ghost and cross-file cases-making nearly one-quarter of bug-inducing commits inherently untraceable. Moreover, current approaches follow fixed pipelines that restrict iterative reasoning and exploration, unlike developers who investigate bugs through an interactive, multi-tool process. To address these challenges, we propose AgentSZZ, an agent-based framework that leverages LLM-driven agents to explore repositories and identify bug-inducing commits. Unlike prior methods, AgentSZZ integrates task-specific tools, domain knowledge, and a ReAct-style loop to enable adaptive and causal tracing of bugs. A structured compression module further improves efficiency by reducing redundant context while preserving key evidence. Extensive experiments on three widely used datasets show that AgentSZZ consistently outperforms state-of-the-art SZZ algorithms across all settings, achieving F1-score gains of up to 27.2% over prior LLM-based approaches. The improvements are especially pronounced in challenging scenarios such as cross-file and ghost commits, with recall gains of up to 300% and 60%, respectively. Ablation studies show that task-specific tools and domain knowledge are critical, while compression tool outputs reduce token consumption by over 30% with negligible impact. The replication package is available.