Efficient Reachability Analysis of Closed-Loop Systems with Neural Network Controllers
This work provides a more efficient and less conservative method for formally analyzing the safety properties of robotic systems controlled by neural networks, which is important for deploying such systems in safety-critical applications.
This paper addresses the challenge of efficiently estimating the forward reachable set of closed-loop systems controlled by neural networks, which is crucial for verifying safety properties. The authors propose a convex optimization approach that, despite being less tight than prior methods, achieves a 10x reduction in conservatism in half the computation time compared to the state-of-the-art by leveraging input set partitioning.
Neural Networks (NNs) can provide major empirical performance improvements for robotic systems, but they also introduce challenges in formally analyzing those systems' safety properties. In particular, this work focuses on estimating the forward reachable set of closed-loop systems with NN controllers. Recent work provides bounds on these reachable sets, yet the computationally efficient approaches provide overly conservative bounds (thus cannot be used to verify useful properties), whereas tighter methods are too intensive for online computation. This work bridges the gap by formulating a convex optimization problem for reachability analysis for closed-loop systems with NN controllers. While the solutions are less tight than prior semidefinite program-based methods, they are substantially faster to compute, and some of the available computation time can be used to refine the bounds through input set partitioning, which more than overcomes the tightness gap. The proposed framework further considers systems with measurement and process noise, thus being applicable to realistic systems with uncertainty. Finally, numerical comparisons show $10\times$ reduction in conservatism in $\frac{1}{2}$ of the computation time compared to the state-of-the-art, and the ability to handle various sources of uncertainty is highlighted on a quadrotor model.