Training Models to Detect Successive Robot Errors from Human Reactions
This work addresses the challenge of improving human-robot interaction by enabling robots to recognize repeated failures through human cues, though it is incremental as it builds on prior studies of human reactions to errors.
The research tackled the problem of detecting successive robot errors by analyzing human reactions, achieving 93.5% accuracy for error detection and 84.1% for classifying successive failures.
As robots become more integrated into society, detecting robot errors is essential for effective human-robot interaction (HRI). When a robot fails repeatedly, how can it know when to change its behavior? Humans naturally respond to robot errors through verbal and nonverbal cues that intensify over successive failures-from confusion and subtle speech changes to visible frustration and impatience. While prior work shows that human reactions can indicate robot failures, few studies examine how these evolving responses reveal successive failures. This research uses machine learning to recognize stages of robot failure from human reactions. In a study with 26 participants interacting with a robot that made repeated conversational errors, behavioral features were extracted from video data to train models for individual users. The best model achieved 93.5% accuracy for detecting errors and 84.1% for classifying successive failures. Modeling the progression of human reactions enhances error detection and understanding of repeated interaction breakdowns in HRI.