LLM Test Generation via Iterative Hybrid Program Analysis
This addresses the problem of low test coverage in real-world software projects for developers, though it is incremental as it builds on existing LLM-based approaches.
The paper tackled the challenge of automating unit test generation for complex methods by introducing Panta, a technique that integrates static control flow and dynamic coverage analysis to guide LLMs iteratively, resulting in 26% higher line coverage and 23% higher branch coverage compared to state-of-the-art methods.
Automating unit test generation remains a significant challenge, particularly for complex methods in real-world projects. While Large Language Models (LLMs) have made strides in code generation, they struggle to achieve high branch coverage due to their limited ability to reason about intricate control flow structures. To address this limitation, we introduce Panta, a technique that emulates the iterative process human developers follow when analyzing code and constructing test cases. Panta integrates static control flow analysis and dynamic code coverage analysis to systematically guide LLMs in identifying uncovered execution paths and generating better test cases. By incorporating an iterative feedback-driven mechanism, our technique continuously refines test generation based on static and dynamic path coverage insights, ensuring more comprehensive and effective testing. Our empirical evaluation, conducted on classes with high cyclomatic complexity from open-source projects, demonstrates that Panta achieves 26% higher line coverage and 23% higher branch coverage compared to the state-of-the-art.