Fighting AI with AI: Leveraging Foundation Models for Assuring AI-Enabled Safety-Critical Systems
This addresses the problem of AI assurance for safety-critical systems like aerospace and autonomous vehicles, presenting a novel approach but with incremental components.
The paper tackles the challenge of assuring AI components in safety-critical systems by leveraging foundation models to bridge the gap between informal requirements and formal specifications, and to test perception systems using human-understandable concepts, though no concrete numbers are provided.
The integration of AI components, particularly Deep Neural Networks (DNNs), into safety-critical systems such as aerospace and autonomous vehicles presents fundamental challenges for assurance. The opacity of AI systems, combined with the semantic gap between high-level requirements and low-level network representations, creates barriers to traditional verification approaches. These AI-specific challenges are amplified by longstanding issues in Requirements Engineering, including ambiguity in natural language specifications and scalability bottlenecks in formalization. We propose an approach that leverages AI itself to address these challenges through two complementary components. REACT (Requirements Engineering with AI for Consistency and Testing) employs Large Language Models (LLMs) to bridge the gap between informal natural language requirements and formal specifications, enabling early verification and validation. SemaLens (Semantic Analysis of Visual Perception using large Multi-modal models) utilizes Vision Language Models (VLMs) to reason about, test, and monitor DNN-based perception systems using human-understandable concepts. Together, these components provide a comprehensive pipeline from informal requirements to validated implementations.