Alert of the Second Decision-maker: An Introduction to Human-AI Conflict
This addresses a critical safety and reliability issue for operators and designers of human-AI collaborative systems, though it is incremental as it builds on existing fault analysis frameworks.
The paper tackles the problem of human-AI conflicts in collaborative systems, which are often overlooked, and systematically introduces the concept, causes, measurement methods, and risk assessment, highlighting that this conflict is a significant and unique risk in digitalized processes.
The collaboration between humans and artificial intelligence (AI) is a significant feature in this digital age. However, humans and AI may have observation, interpretation, and action conflicts when working synchronously. This phenomenon is often masked by faults and, unfortunately, overlooked. This paper systematically introduces the human-AI conflict concept, causes, measurement methods, and risk assessment. The results highlight that there is a potential second decision-maker besides the human, which is the AI; the human-AI conflict is a unique and emerging risk in digitalized process systems; and this is an interdisciplinary field that needs to be distinguished from traditional fault and failure analysis; the conflict risk is significant and cannot be ignored.