The Black-Box Simplex Architecture for Runtime Assurance of Autonomous CPS
This addresses safety assurance for autonomous systems using complex controllers like neural networks, which are difficult to verify statically, but is incremental as it builds on the existing Simplex Architecture framework.
The paper tackles the problem of ensuring safety in autonomous cyber-physical systems by proposing the Black-Box Simplex Architecture, which uses runtime checks to allow switching to a backup controller without requiring static verification, and demonstrates its effectiveness in case studies including preventing collisions in F-16 aircraft groups.
The Simplex Architecture is a runtime assurance framework where control authority may switch from an unverified and potentially unsafe advanced controller to a backup baseline controller in order to maintain the safety of an autonomous cyber-physical system. In this work, we show that runtime checks can replace the requirement to statically verify safety of the baseline controller. This is important as there are many powerful control techniques, such as model-predictive control and neural network controllers, that work well in practice but are difficult to statically verify. Since the method does not use internal information about the advanced or baseline controller, we call the approach the Black-Box Simplex Architecture. We prove the architecture is safe and present two case studies where (i) model-predictive control provides safe multi-robot coordination, and (ii) neural networks provably prevent collisions in groups of F-16 aircraft, despite the controllers occasionally outputting unsafe commands.