Echo: Graph-Enhanced Retrieval and Execution Feedback for Issue Reproduction Test Generation
This work addresses the problem of time-consuming manual test case creation for developers by automating the generation of bug-reproducing tests, offering an incremental improvement over existing methods.
This paper introduces Echo, an agent that generates issue-reproducing test cases to help developers identify bug root causes. Echo achieves a 66.28% success rate on the SWT-Bench Verified dataset, setting a new state of the art among open-source approaches.
Identifying the root cause of a bug remains difficult for many developers because bug reports often lack a bug reproducing test case that reliably triggers the failure. Manually writing such test cases is time-consuming and requires substantial effort to understand the codebase and isolate the failing behavior. To address this challenge, we propose Echo, an agent for generating issue reproducing test cases, which advances previous work in several ways. During generation, Echo strengthens context retrieval by leveraging a code graph and a novel automatic query-refinement strategy. Echo also improves upon previous tools by automatically executing generated test cases, a first-of-its-kind feature that seamlessly integrates into practical development workflows. In addition, Echo generates potential patches and uses the patched version to validate whether a candidate test meets the fail-to-pass criterion and to provide actionable feedback for refinement. Unlike prior bug-reproduction agents that sample and rank multiple candidate tests, Echo generates a single test per issue, offering a better cost-performance trade-off. Experiments on SWT-Bench Verified show that Echo establishes a new state of the art among open-source approaches, achieving a 66.28% success rate.