Explainable Abuse Detection as Intent Classification and Slot Filling
This work addresses the need for explainable and consistent abuse detection for social media platform moderators, though it is incremental in adapting existing intent classification and slot filling methods to this domain.
The paper tackles the problem of unreliable abuse detection in social media by introducing policy-aware detection, which uses a machine-friendly representation of moderation policies as intents and slots, and demonstrates this approach on a dataset of 3,535 English posts while providing rationales for model decisions.
To proactively offer social media users a safe online experience, there is a need for systems that can detect harmful posts and promptly alert platform moderators. In order to guarantee the enforcement of a consistent policy, moderators are provided with detailed guidelines. In contrast, most state-of-the-art models learn what abuse is from labelled examples and as a result base their predictions on spurious cues, such as the presence of group identifiers, which can be unreliable. In this work we introduce the concept of policy-aware abuse detection, abandoning the unrealistic expectation that systems can reliably learn which phenomena constitute abuse from inspecting the data alone. We propose a machine-friendly representation of the policy that moderators wish to enforce, by breaking it down into a collection of intents and slots. We collect and annotate a dataset of 3,535 English posts with such slots, and show how architectures for intent classification and slot filling can be used for abuse detection, while providing a rationale for model decisions.