AutoSpec: Automated Generation of Neural Network Specifications
This addresses a bottleneck in neural network safety verification for learning-augmented systems, though it appears incremental as it builds on existing verification practices.
The paper tackles the problem of manually defining neural network specifications for formal verification by introducing AutoSpec, the first framework for automated specification generation. Their evaluation shows AutoSpec outperforms human-defined specifications and baseline approaches across four applications.
The increasing adoption of neural networks in learning-augmented systems highlights the importance of model safety and robustness, particularly in safety-critical domains. Despite progress in the formal verification of neural networks, current practices require users to manually define model specifications -- properties that dictate expected model behavior in various scenarios. This manual process, however, is prone to human error, limited in scope, and time-consuming. In this paper, we introduce AutoSpec, the first framework to automatically generate comprehensive and accurate specifications for neural networks in learning-augmented systems. We also propose the first set of metrics for assessing the accuracy and coverage of model specifications, establishing a benchmark for future comparisons. Our evaluation across four distinct applications shows that AutoSpec outperforms human-defined specifications as well as two baseline approaches introduced in this study.