Trouble on the Horizon: Forecasting the Derailment of Online Conversations as they Develop
This addresses the need for timely moderation in online discussions to prevent antisocial behavior, representing a novel forecasting approach rather than incremental detection.
The paper tackled the problem of forecasting when online conversations will derail into toxic exchanges before they actually turn toxic, by modeling derailment as an emerging property of conversational dynamics, and showed that their model outperforms state-of-the-art systems on two new datasets.
Online discussions often derail into toxic exchanges between participants. Recent efforts mostly focused on detecting antisocial behavior after the fact, by analyzing single comments in isolation. To provide more timely notice to human moderators, a system needs to preemptively detect that a conversation is heading towards derailment before it actually turns toxic. This means modeling derailment as an emerging property of a conversation rather than as an isolated utterance-level event. Forecasting emerging conversational properties, however, poses several inherent modeling challenges. First, since conversations are dynamic, a forecasting model needs to capture the flow of the discussion, rather than properties of individual comments. Second, real conversations have an unknown horizon: they can end or derail at any time; thus a practical forecasting model needs to assess the risk in an online fashion, as the conversation develops. In this work we introduce a conversational forecasting model that learns an unsupervised representation of conversational dynamics and exploits it to predict future derailment as the conversation develops. By applying this model to two new diverse datasets of online conversations with labels for antisocial events, we show that it outperforms state-of-the-art systems at forecasting derailment.