WHoW: A Cross-domain Approach for Analysing Conversation Moderation
This work addresses the need for systematic analysis of moderation behavior to enhance understanding and develop moderator agents, but it is incremental as it builds on existing evaluation frameworks.
The authors tackled the problem of analyzing conversation moderation across different domains by proposing the WHoW framework, which evaluates moderators' motives, dialogue acts, and target speakers, and they annotated over 20,000 sentences to demonstrate its cross-domain generalizability and reveal distinct strategies between TV debates and radio panel discussions.
We propose WHoW, an evaluation framework for analyzing the facilitation strategies of moderators across different domains/scenarios by examining their motives (Why), dialogue acts (How) and target speaker (Who). Using this framework, we annotated 5,657 moderation sentences with human judges and 15,494 sentences with GPT-4o from two domains: TV debates and radio panel discussions. Comparative analysis demonstrates the framework's cross-domain generalisability and reveals distinct moderation strategies: debate moderators emphasise coordination and facilitate interaction through questions and instructions, while panel discussion moderators prioritize information provision and actively participate in discussions. Our analytical framework works for different moderation scenarios, enhances our understanding of moderation behaviour through automatic large-scale analysis, and facilitates the development of moderator agents.