FuzzAug: Data Augmentation by Coverage-guided Fuzzing for Neural Test Generation
This work addresses the problem of automated test case generation for software engineering, particularly for newer programming languages, by enhancing neural methods with dynamic analysis insights, though it is incremental in integrating existing fuzzing techniques.
The paper tackles the limited diversity and volume of unit test datasets for neural test generation by introducing FuzzAug, a data augmentation technique that combines coverage-guided fuzzing with large language models, resulting in a doubled training dataset size and significant performance improvements over baselines.
Testing is essential to modern software engineering for building reliable software. Given the high costs of manually creating test cases, automated test case generation, particularly methods utilizing large language models, has become increasingly popular. These neural approaches generate semantically meaningful tests that are more maintainable compared with traditional automatic testing methods like fuzzing. However, the diversity and volume of unit tests in current datasets are limited, especially for newer but important languages. In this paper, we present a novel data augmentation technique, FuzzAug, that introduces the benefits of fuzzing to large language models by introducing valid testing semantics and providing diverse coverage-guided inputs. Doubling the size of training datasets, FuzzAug improves the performance from the baselines significantly. This technique demonstrates the potential of introducing prior knowledge from dynamic software analysis to improve neural test generation, offering significant enhancements in neural test generation.