Are you still with me? Continuous Engagement Assessment from a Robot's Point of View
This work addresses the challenge of generic engagement assessment for robots in HRI, enabling applications like in-situ reinforcement learning and interaction optimization, though it is incremental in building upon intuitive human assessment methods.
The paper tackled the problem of continuously measuring user engagement with robots in Human-Robot Interaction (HRI) by proposing a novel regression model using CNN and LSTM networks to compute a single scalar engagement score from video streams. The model was trained on a dataset from a museum tour guide robot and successfully transferred to a different dataset, demonstrating its effectiveness across varied settings.
Continuously measuring the engagement of users with a robot in a Human-Robot Interaction (HRI) setting paves the way towards in-situ reinforcement learning, improve metrics of interaction quality, and can guide interaction design and behaviour optimisation. However, engagement is often considered very multi-faceted and difficult to capture in a workable and generic computational model that can serve as an overall measure of engagement. Building upon the intuitive ways humans successfully can assess situation for a degree of engagement when they see it, we propose a novel regression model (utilising CNN and LSTM networks) enabling robots to compute a single scalar engagement during interactions with humans from standard video streams, obtained from the point of view of an interacting robot. The model is based on a long-term dataset from an autonomous tour guide robot deployed in a public museum, with continuous annotation of a numeric engagement assessment by three independent coders. We show that this model not only can predict engagement very well in our own application domain but show its successful transfer to an entirely different dataset (with different tasks, environment, camera, robot and people). The trained model and the software is available to the HRI community as a tool to measure engagement in a variety of settings.