A Study on Domain Generalization for Failure Detection through Human Reactions in HRI
This work addresses the problem of model generalization in HRI for failure detection, but it is incremental as it primarily analyzes existing performance drops without introducing new methods.
The study investigated domain generalization for failure detection models in human-robot interaction by training deep learning models on human facial expression data from two distinct datasets (controlled lab and online). When tested on the alternate dataset, the models showed a significant performance drop, highlighting robustness issues.
Machine learning models are commonly tested in-distribution (same dataset); performance almost always drops in out-of-distribution settings. For HRI research, the goal is often to develop generalized models. This makes domain generalization - retaining performance in different settings - a critical issue. In this study, we present a concise analysis of domain generalization in failure detection models trained on human facial expressions. Using two distinct datasets of humans reacting to videos where error occurs, one from a controlled lab setting and another collected online, we trained deep learning models on each dataset. When testing these models on the alternate dataset, we observed a significant performance drop. We reflect on the causes for the observed model behavior and leave recommendations. This work emphasizes the need for HRI research focusing on improving model robustness and real-life applicability.