Reliability, Resilience and Human Factors Engineering for Trustworthy AI Systems
This work addresses the need for trustworthy AI systems in critical operations across industries, though it is incremental as it adapts existing engineering methods to AI.
The paper tackles the problem of ensuring reliability and safety in AI systems by integrating established reliability and resilience engineering principles, proposing a framework that adapts classical methods like failure rate and MTBF to AI, and demonstrates its applicability using real-world data from platforms such as OpenAI.
As AI systems become integral to critical operations across industries and services, ensuring their reliability and safety is essential. We offer a framework that integrates established reliability and resilience engineering principles into AI systems. By applying traditional metrics such as failure rate and Mean Time Between Failures (MTBF) along with resilience engineering and human reliability analysis, we propose an integrate framework to manage AI system performance, and prevent or efficiently recover from failures. Our work adapts classical engineering methods to AI systems and outlines a research agenda for future technical studies. We apply our framework to a real-world AI system, using system status data from platforms such as openAI, to demonstrate its practical applicability. This framework aligns with emerging global standards and regulatory frameworks, providing a methodology to enhance the trustworthiness of AI systems. Our aim is to guide policy, regulation, and the development of reliable, safe, and adaptable AI technologies capable of consistent performance in real-world environments.