How Did We Get Here? Summarizing Conversation Dynamics
This work addresses the need for better tools to analyze and predict conversation trajectories, particularly for detecting toxicity, but it is incremental as it builds on existing forecasting tasks with new summarization methods.
The paper tackles the problem of summarizing conversation dynamics to capture how interactions evolve, showing that these summaries help humans predict toxic behavior three times faster and with greater confidence, and improve automated forecasting accuracy compared to using raw transcripts.
Throughout a conversation, the way participants interact with each other is in constant flux: their tones may change, they may resort to different strategies to convey their points, or they might alter their interaction patterns. An understanding of these dynamics can complement that of the actual facts and opinions discussed, offering a more holistic view of the trajectory of the conversation: how it arrived at its current state and where it is likely heading. In this work, we introduce the task of summarizing the dynamics of conversations, by constructing a dataset of human-written summaries, and exploring several automated baselines. We evaluate whether such summaries can capture the trajectory of conversations via an established downstream task: forecasting whether an ongoing conversation will eventually derail into toxic behavior. We show that they help both humans and automated systems with this forecasting task. Humans make predictions three times faster, and with greater confidence, when reading the summaries than when reading the transcripts. Furthermore, automated forecasting systems are more accurate when constructing, and then predicting based on, summaries of conversation dynamics, compared to directly predicting on the transcripts.