Effectively Detecting and Responding to Online Harassment with Large Language Models
This addresses online harassment for users of private messaging platforms like Instagram, though it is incremental as it applies existing LLM methods to a new domain.
The researchers tackled online harassment detection and response generation on Instagram's private messaging platform by using a Large Language Model (LLM) pipeline for labeling and simulating responses, finding that the LLM effectively identifies harassment and generates responses superior in helpfulness compared to human ones.
Online harassment has been a persistent issue in the online space. Predominantly, research focused on online harassment in public social media platforms, while less is placed on private messaging platforms. To address online harassment on one private messaging platform, Instagram, we leverage the capabilities of Large Language Models (LLMs). To achieve this, we recruited human labelers to identify online harassment in an Instagram messages dataset. Using the previous conversation as context, we utilize an LLM pipeline to conduct large-scale labeling on Instagram messages and evaluate its performance against human labels. Then, we use LLM to generate and evaluate simulated responses to online harassment messages. We find that the LLM labeling pipeline is capable of identifying online harassment in private messages. By comparing human responses and simulated responses, we also demonstrate that our simulated responses are superior in helpfulness compared to original human responses.