Gray-box Adversarial Testing for Control Systems with Machine Learning Component
This addresses the problem of ensuring safety in control systems with machine learning components, which is crucial for industry adoption but currently hindered by verification difficulties, representing an incremental improvement in adversarial testing methods.
The paper tackles the challenge of testing and verifying neural network-based controllers in safety-critical systems by introducing a gradient-based method to find adversarial samples against system-level requirements, showing that it outperforms Simulated Annealing optimization in experiments.
Neural Networks (NN) have been proposed in the past as an effective means for both modeling and control of systems with very complex dynamics. However, despite the extensive research, NN-based controllers have not been adopted by the industry for safety critical systems. The primary reason is that systems with learning based controllers are notoriously hard to test and verify. Even harder is the analysis of such systems against system-level specifications. In this paper, we provide a gradient based method for searching the input space of a closed-loop control system in order to find adversarial samples against some system-level requirements. Our experimental results show that combined with randomized search, our method outperforms Simulated Annealing optimization.