Sustainability Analysis of Prompt Strategies for SLM-based Automated Test Generation
This work addresses sustainability issues in AI-driven software testing for developers and researchers, but it is incremental as it extends existing studies to prompt strategies for SLMs.
This paper tackles the problem of evaluating the computational and environmental sustainability of prompt engineering strategies for automated test generation using Small Language Models, finding that prompt strategies significantly impact sustainability outcomes, with reasoning-intensive methods like Chain of Thought achieving higher coverage but incurring higher costs, while simpler strategies like Zero-Shot offer competitive coverage with lower environmental impact.
The growing adoption of prompt-based automation in software testing raises important issues regarding its computational and environmental sustainability. Existing sustainability studies in AI-driven testing primarily focus on large language models, leaving the impact of prompt engineering strategies largely unexplored - particularly in the context of Small Language Models (SLMs). This gap is critical, as prompt design directly influences inference behavior, execution cost, and resource utilization, even when model size is fixed. To the best of our knowledge, this paper presents the first systematic sustainability evaluation of prompt engineering strategies for automated test generation using SLMs. We analyze seven prompt strategies across three open-source SLMs under a controlled experimental setup. Our evaluation jointly considers execution time, token usage, energy consumption, carbon emissions, and coverage test quality, the latter assessed through coverage analysis of the generated test scripts. The results show that prompt strategies have a substantial and independent impact on sustainability outcomes, often outweighing the effect of model choice. Reasoning intensive strategies such as Chain of Thought and Self-Consistency achieve higher coverage but incur significantly higher execution time, energy consumption, and carbon emissions. In contrast, simpler strategies such as Zero-Shot and ReAct deliver competitive coverage test quality with markedly lower environmental cost, while Least-to-Most and Program of Thought offer balanced trade-offs.