Dynamic Forecasting of Conversation Derailment
This work addresses the need for early moderation in public online conversations, but it is incremental as it builds on limited prior work with mixed results.
The paper tackles the problem of forecasting derailment in online conversations by applying a pretrained language encoder and shifting from static to dynamic training, achieving a longer forecast horizon with a small F1 drop in high-quality data but detrimental performance in low-quality data.
Online conversations can sometimes take a turn for the worse, either due to systematic cultural differences, accidental misunderstandings, or mere malice. Automatically forecasting derailment in public online conversations provides an opportunity to take early action to moderate it. Previous work in this space is limited, and we extend it in several ways. We apply a pretrained language encoder to the task, which outperforms earlier approaches. We further experiment with shifting the training paradigm for the task from a static to a dynamic one to increase the forecast horizon. This approach shows mixed results: in a high-quality data setting, a longer average forecast horizon can be achieved at the cost of a small drop in F1; in a low-quality data setting, however, dynamic training propagates the noise and is highly detrimental to performance.