Analyzing Norm Violations in Live-Stream Chat
This work addresses the challenge of maintaining safe online communities on live-streaming platforms like Twitch and YouTube Live, which is incremental as it adapts existing NLP methods to a new domain.
The paper tackles the problem of detecting toxic language and norm violations in live-stream chats, where existing models perform poorly due to unique data characteristics like limited comment visibility and lack of thread structure. By defining norm violation categories, annotating 4,583 Twitch comments, and training models with contextual information identified from a user study, the results show a 35% boost in moderation performance.
Toxic language, such as hate speech, can deter users from participating in online communities and enjoying popular platforms. Previous approaches to detecting toxic language and norm violations have been primarily concerned with conversations from online forums and social media, such as Reddit and Twitter. These approaches are less effective when applied to conversations on live-streaming platforms, such as Twitch and YouTube Live, as each comment is only visible for a limited time and lacks a thread structure that establishes its relationship with other comments. In this work, we share the first NLP study dedicated to detecting norm violations in conversations on live-streaming platforms. We define norm violation categories in live-stream chats and annotate 4,583 moderated comments from Twitch. We articulate several facets of live-stream data that differ from other forums, and demonstrate that existing models perform poorly in this setting. By conducting a user study, we identify the informational context humans use in live-stream moderation, and train models leveraging context to identify norm violations. Our results show that appropriate contextual information can boost moderation performance by 35\%.