VERIFAI: A Toolkit for the Design and Analysis of Artificial Intelligence-Based Systems
This toolkit addresses verification problems for developers of AI-based systems, but it appears incremental as it builds on existing formal methods approaches.
The authors tackled the challenge of applying formal methods to AI/ML systems by developing VERIFAI, a toolkit for design and analysis, which includes features like temporal-logic falsification and model-based fuzz testing.
We present VERIFAI, a software toolkit for the formal design and analysis of systems that include artificial intelligence (AI) and machine learning (ML) components. VERIFAI particularly seeks to address challenges with applying formal methods to perception and ML components, including those based on neural networks, and to model and analyze system behavior in the presence of environment uncertainty. We describe the initial version of VERIFAI which centers on simulation guided by formal models and specifications. Several use cases are illustrated with examples, including temporal-logic falsification, model-based systematic fuzz testing, parameter synthesis, counterexample analysis, and data set augmentation.