Static Program Analysis Guided LLM Based Unit Test Generation
This addresses scalability and sample usage issues in automated test generation for Java developers, but is incremental as it builds on existing LLM-based methods.
The paper tackles the problem of generating unit tests for Java methods using LLMs by augmenting prompts with concise context from program analysis, which improved effectiveness on commercial and open-source projects.
We describe a novel approach to automating unit test generation for Java methods using large language models (LLMs). Existing LLM-based approaches rely on sample usage(s) of the method to test (focal method) and/or provide the entire class of the focal method as input prompt and context. The former approach is often not viable due to the lack of sample usages, especially for newly written focal methods. The latter approach does not scale well enough; the bigger the complexity of the focal method and larger associated class, the harder it is to produce adequate test code (due to factors such as exceeding the prompt and context lengths of the underlying LLM). We show that augmenting prompts with \emph{concise} and \emph{precise} context information obtained by program analysis %of the focal method increases the effectiveness of generating unit test code through LLMs. We validate our approach on a large commercial Java project and a popular open-source Java project.