HPCAgentTester: A Multi-Agent LLM Approach for Enhanced HPC Unit Test Generation
This addresses the problem of unreliable unit testing for HPC software developers, offering a more robust solution, though it appears incremental as it builds on existing LLM methods with a specialized multi-agent approach.
The paper tackles the challenge of unit testing in High-Performance Computing (HPC) by introducing HPCAgentTester, a multi-agent LLM framework that automates test generation for OpenMP and MPI software, resulting in significantly improved test compilation rates and correctness compared to standalone LLMs.
Unit testing in High-Performance Computing (HPC) is critical but challenged by parallelism, complex algorithms, and diverse hardware. Traditional methods often fail to address non-deterministic behavior and synchronization issues in HPC applications. This paper introduces HPCAgentTester, a novel multi-agent Large Language Model (LLM) framework designed to automate and enhance unit test generation for HPC software utilizing OpenMP and MPI. HPCAgentTester employs a unique collaborative workflow where specialized LLM agents (Recipe Agent and Test Agent) iteratively generate and refine test cases through a critique loop. This architecture enables the generation of context-aware unit tests that specifically target parallel execution constructs, complex communication patterns, and hierarchical parallelism. We demonstrate HPCAgentTester's ability to produce compilable and functionally correct tests for OpenMP and MPI primitives, effectively identifying subtle bugs that are often missed by conventional techniques. Our evaluation shows that HPCAgentTester significantly improves test compilation rates and correctness compared to standalone LLMs, offering a more robust and scalable solution for ensuring the reliability of parallel software systems.