Formal Verification of End-to-End Learning in Cyber-Physical Systems: Progress and Challenges
This work tackles the critical safety problem for autonomous systems, but it is incremental as it focuses on identifying limitations and early progress rather than presenting a novel solution.
The paper addresses the challenge of providing strong safety guarantees for autonomous systems like self-driving cars and drones through formal verification, highlighting that existing techniques are limited by three key assumptions and summarizing preliminary efforts to enhance verification evidence.
Autonomous systems -- such as self-driving cars, autonomous drones, and automated trains -- must come with strong safety guarantees. Over the past decade, techniques based on formal methods have enjoyed some success in providing strong correctness guarantees for large software systems including operating system kernels, cryptographic protocols, and control software for drones. These successes suggest it might be possible to ensure the safety of autonomous systems by constructing formal, computer-checked correctness proofs. This paper identifies three assumptions underlying existing formal verification techniques, explains how each of these assumptions limits the applicability of verification in autonomous systems, and summarizes preliminary work toward improving the strength of evidence provided by formal verification.