Speech Is Not Enough: Interpreting Nonverbal Indicators of Common Knowledge and Engagement
This addresses the need for better group dynamics support in educational settings, but it appears incremental as it builds on existing multimodal analytics without introducing a new method.
The paper tackled the problem of understanding group collaboration by developing an AI Partner that uses multimodal analytics to detect nonverbal behavior in student interactions, aiming to track common ground and engagement, but no concrete results or numbers are provided.
Our goal is to develop an AI Partner that can provide support for group problem solving and social dynamics. In multi-party working group environments, multimodal analytics is crucial for identifying non-verbal interactions of group members. In conjunction with their verbal participation, this creates an holistic understanding of collaboration and engagement that provides necessary context for the AI Partner. In this demo, we illustrate our present capabilities at detecting and tracking nonverbal behavior in student task-oriented interactions in the classroom, and the implications for tracking common ground and engagement.