Formalizing and Evaluating Requirements of Perception Systems for Automated Vehicles using Spatio-Temporal Perception Logic
This work addresses the need for more robust evaluation methods for perception systems in automated vehicles, though it is incremental as it builds on existing logic-based approaches.
The paper tackles the problem of evaluating perception systems for automated vehicles by introducing Spatio-Temporal Perception Logic (STPL), which enables reasoning over spatial and temporal data to assess performance beyond frame-by-frame metrics, and identifies an efficiently monitorable fragment for offline analysis.
Automated vehicles (AV) heavily depend on robust perception systems. Current methods for evaluating vision systems focus mainly on frame-by-frame performance. Such evaluation methods appear to be inadequate in assessing the performance of a perception subsystem when used within an AV. In this paper, we present a logic -- referred to as Spatio-Temporal Perception Logic (STPL) -- which utilizes both spatial and temporal modalities. STPL enables reasoning over perception data using spatial and temporal operators. One major advantage of STPL is that it facilitates basic sanity checks on the functional performance of the perception system, even without ground-truth data in some cases. We identify a fragment of STPL which is efficiently monitorable offline in polynomial time. Finally, we present a range of specifications for AV perception systems to highlight the types of requirements that can be expressed and analyzed through offline monitoring with STPL.