Program Structure-aware Language Models: Targeted Software Testing beyond Textual Semantics
For software testing practitioners, GLMTest addresses the bottleneck of steering LLMs toward high-risk branches to improve bug and vulnerability discovery.
GLMTest, a program structure-aware LLM framework, integrates code property graphs and code semantics to enable targeted test case generation for specific execution branches, improving branch accuracy from 27.4% to 50.2% on the TestGenEval benchmark compared to state-of-the-art LLMs.
Recent advances in large language models for test case generation have improved branch coverage via prompt-engineered mutations. However, they still lack principled mechanisms for steering models toward specific high-risk execution branches, limiting their effectiveness for discovering subtle bugs and security vulnerabilities. We propose GLMTest, the first program structure-aware LLM framework for targeted test case generation that seamlessly integrates code property graphs and code semantics using a graph neural network and a language model to condition test case generation on execution branches. This structured conditioning enables controllable and branch-targeted test case generation, thereby potentially enhancing bug and security risk discovery. Experiments on real-world projects show that GLMTest built on a Qwen2.5-Coder-7B-Instruct model improves branch accuracy from 27.4% to 50.2% on TestGenEval benchmark compared with state-of-the-art LLMs, i.e., Claude-Sonnet-4.5 and GPT-4o-mini.