HySafe-AI: Hybrid Safety Architectural Analysis Framework for AI Systems: A Case Study
This work addresses safety analysis for AI systems in critical domains like autonomous driving, but it appears incremental as it adapts existing methods rather than proposing a fundamentally new approach.
The paper tackles the challenge of evaluating safety in AI systems with complex architectures like large language models, by introducing HySAFE-AI, a hybrid framework that adapts traditional safety analysis methods such as FMEA and FTA to improve their efficacy for these systems.
AI has become integral to safety-critical areas like autonomous driving systems (ADS) and robotics. The architecture of recent autonomous systems are trending toward end-to-end (E2E) monolithic architectures such as large language models (LLMs) and vision language models (VLMs). In this paper, we review different architectural solutions and then evaluate the efficacy of common safety analyses such as failure modes and effect analysis (FMEA) and fault tree analysis (FTA). We show how these techniques can be improved for the intricate nature of the foundational models, particularly in how they form and utilize latent representations. We introduce HySAFE-AI, Hybrid Safety Architectural Analysis Framework for AI Systems, a hybrid framework that adapts traditional methods to evaluate the safety of AI systems. Lastly, we offer hints of future work and suggestions to guide the evolution of future AI safety standards.