Something Just Like TRuST : Toxicity Recognition of Span and Target
This work addresses toxicity recognition for online content and language models, but it is incremental as it builds on existing datasets and methods.
The paper tackles toxicity detection in online content by introducing TRuST, a comprehensive dataset with labels for toxicity, target social group, and toxic spans, and benchmarks state-of-the-art large language models, finding that fine-tuned models outperform zero-shot and few-shot prompting but performance remains low for certain social groups.
Toxicity in online content, including content generated by language models, has become a critical concern due to its potential for negative psychological and social impact. This paper introduces TRuST, a comprehensive dataset designed to improve toxicity detection that merges existing datasets, and has labels for toxicity, target social group, and toxic spans. It includes a diverse range of target groups such as ethnicity, gender, religion, disability, and politics, with both human/machine-annotated and human machine-generated data. We benchmark state-of-the-art large language models (LLMs) on toxicity detection, target group identification, and toxic span extraction. We find that fine-tuned models consistently outperform zero-shot and few-shot prompting, though performance remains low for certain social groups. Further, reasoning capabilities do not significantly improve performance, indicating that LLMs have weak social reasoning skills.