Refining Perception Contracts: Case Studies in Vision-based Safe Auto-landing
This work addresses safety assurance for autonomous flight systems using ML perception, representing an incremental advancement in contract-based testing methodology.
The paper tackled the problem of ensuring safety in vision-based control systems for aircraft and drones by refining perception contracts, resulting in testable contracts that establish safe landing and gate-passing conditions and identify potential safety violations like low-horizon sun.
Perception contracts provide a method for evaluating safety of control systems that use machine learning for perception. A perception contract is a specification for testing the ML components, and it gives a method for proving end-to-end system-level safety requirements. The feasibility of contract-based testing and assurance was established earlier in the context of straight lane keeping: a 3-dimensional system with relatively simple dynamics. This paper presents the analysis of two 6 and 12-dimensional flight control systems that use multi-stage, heterogeneous, ML-enabled perception. The paper advances methodology by introducing an algorithm for constructing data and requirement guided refinement of perception contracts (DaRePC). The resulting analysis provides testable contracts which establish the state and environment conditions under which an aircraft can safety touchdown on the runway and a drone can safely pass through a sequence of gates. It can also discover conditions (e.g., low-horizon sun) that can possibly violate the safety of the vision-based control system.