Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems
This work provides a foundational framework for researchers and practitioners designing, implementing, and evaluating human oversight in high-risk AI systems, addressing a critical gap in the field.
The paper addresses the lack of a common foundational understanding of human oversight of AI systems by proposing a practical framework based on cross-disciplinary insights. It provides a working definition, architecture, and processes for effective oversight, along with a documentation template and open research challenges.
The use of Artificial Intelligence (AI) in high-risk, decision-making scenarios presents technical, safety, and normative challenges; problems that may only be ameliorated by human oversight. However, notions of human oversight lack a common foundational understanding: oversight architectures are not well defined, the roles involved remain unclear, and implementation steps are opaque. Hence, researchers and practitioners struggle to determine how to design, implement, and evaluate systems that enable effective human oversight. This paper advances a practical framework for effective human oversight of AI systems, based on a cross-disciplinary perspective that draws on insights from computer science, human-computer interaction, psychology, philosophy, and law. The core contributions are: (1) a foundational framework, with a working definition, architecture and processes for effective human oversight of AI systems; (2) an initial template for documenting oversight architectures and processes, applied to diverse domains; and (3) a synthesis of open research challenges that need to be considered in the emerging field of effective human oversight of AI systems.