Detecting Interlocutor Confusion in Situated Human-Avatar Dialogue: A Pilot Study
This addresses the problem of improving engagement in conversational systems for users by enabling confusion monitoring, though it is an incremental step as a pilot study.
The study tackled detecting user confusion in human-avatar dialogues by defining a tailored confusion concept and collecting data via a Wizard-of-Oz scenario, finding a significant relationship between deep learning-based indicators (emotion, head pose, eye gaze) and confusion states despite a small pilot group.
In order to enhance levels of engagement with conversational systems, our long term research goal seeks to monitor the confusion state of a user and adapt dialogue policies in response to such user confusion states. To this end, in this paper, we present our initial research centred on a user-avatar dialogue scenario that we have developed to study the manifestation of confusion and in the long term its mitigation. We present a new definition of confusion that is particularly tailored to the requirements of intelligent conversational system development for task-oriented dialogue. We also present the details of our Wizard-of-Oz based data collection scenario wherein users interacted with a conversational avatar and were presented with stimuli that were in some cases designed to invoke a confused state in the user. Post study analysis of this data is also presented. Here, three pre-trained deep learning models were deployed to estimate base emotion, head pose and eye gaze. Despite a small pilot study group, our analysis demonstrates a significant relationship between these indicators and confusion states. We understand this as a useful step forward in the automated analysis of the pragmatics of dialogue.