Learning to Evaluate Autonomous Behaviour in Human-Robot Interaction
This provides a reproducible and systematic evaluation tool for researchers in robotics and human-robot interaction, though it is incremental as it builds on existing imitation learning methods.
The paper tackles the challenge of evaluating autonomous humanoid robot performance in human-robot interaction by proposing a general framework that measures imitation learning methods based on trajectory quality, resulting in a method more aligned with robot success rates than baselines.
Evaluating and comparing the performance of autonomous Humanoid Robots is challenging, as success rate metrics are difficult to reproduce and fail to capture the complexity of robot movement trajectories, critical in Human-Robot Interaction and Collaboration (HRIC). To address these challenges, we propose a general evaluation framework that measures the quality of Imitation Learning (IL) methods by focusing on trajectory performance. We devise the Neural Meta Evaluator (NeME), a deep learning model trained to classify actions from robot joint trajectories. NeME serves as a meta-evaluator to compare the performance of robot control policies, enabling policy evaluation without requiring human involvement in the loop. We validate our framework on ergoCub, a humanoid robot, using teleoperation data and comparing IL methods tailored to the available platform. The experimental results indicate that our method is more aligned with the success rate obtained on the robot than baselines, offering a reproducible, systematic, and insightful means for comparing the performance of multimodal imitation learning approaches in complex HRI tasks.