Collaborative Artificial Intelligence Needs Stronger Assurances Driven by Risks
This work tackles safety assurance for collaborative AI systems, which is crucial for preventing harm to humans in shared environments, but it appears incremental as it builds on existing risk management approaches.
The paper addresses the problem of ensuring safety and compliance in collaborative AI systems (CAISs) that work with humans, highlighting the lack of large-scale impact due to unmanaged risks, and proposes a risk-driven assurance process as a solution.
Collaborative AI systems (CAISs) aim at working together with humans in a shared space to achieve a common goal. This critical setting yields hazardous circumstances that could harm human beings. Thus, building such systems with strong assurances of compliance with requirements, domain-specific standards and regulations is of greatest importance. Only few scale impact has been reported so far for such systems since much work remains to manage possible risks. We identify emerging problems in this context and then we report our vision, as well as the progress of our multidisciplinary research team composed of software/systems, and mechatronics engineers to develop a risk-driven assurance process for CAISs.