Detecting Agreement in Multi-party Conversational AI
This work addresses the challenge of practical usability in multi-party conversational systems for Socially Assistive Robots, though it appears incremental as it focuses on a specific game scenario.
The paper tackled the problem of detecting user agreement or disagreement in a multi-party conversational AI system for a trivia quiz game, achieving results that include performance metrics and user assessments, with annotated transcripts and code released open-source.
Today, conversational systems are expected to handle conversations in multi-party settings, especially within Socially Assistive Robots (SARs). However, practical usability remains difficult as there are additional challenges to overcome, such as speaker recognition, addressee recognition, and complex turn-taking. In this paper, we present our work on a multi-party conversational system, which invites two users to play a trivia quiz game. The system detects users' agreement or disagreement on a final answer and responds accordingly. Our evaluation includes both performance and user assessment results, with a focus on detecting user agreement. Our annotated transcripts and the code for the proposed system have been released open-source on GitHub.