Forecasting Conversation Derailments Through Generation
This addresses the need for better conversation moderation and conflict resolution tools, though it is incremental as it builds on existing prediction methods.
The paper tackles the problem of forecasting conversation derailments by generating multiple future conversation trajectories using a fine-tuned LLM, achieving state-of-the-art results on English benchmarks with significant accuracy gains.
Forecasting conversation derailment can be useful in real-world settings such as online content moderation, conflict resolution, and business negotiations. However, despite language models' success at identifying offensive speech present in conversations, they struggle to forecast future conversation derailments. In contrast to prior work that predicts conversation outcomes solely based on the past conversation history, our approach samples multiple future conversation trajectories conditioned on existing conversation history using a fine-tuned LLM. It predicts the conversation outcome based on the consensus of these trajectories. We also experimented with leveraging socio-linguistic attributes, which reflect turn-level conversation dynamics, as guidance when generating future conversations. Our method of future conversation trajectories surpasses state-of-the-art results on English conversation derailment prediction benchmarks and demonstrates significant accuracy gains in ablation studies.