THOS: A Benchmark Dataset for Targeted Hate and Offensive Speech
This addresses the scarcity of datasets for targeted hate and offensive speech detection, which is an incremental improvement for researchers and practitioners in content moderation.
The authors tackled the problem of detecting harmful content on social media by introducing THOS, a dataset of 8.3k tweets with fine-grained annotations for target classes, enabling the training of classifiers based on Large Language Models.
Detecting harmful content on social media, such as Twitter, is made difficult by the fact that the seemingly simple yes/no classification conceals a significant amount of complexity. Unfortunately, while several datasets have been collected for training classifiers in hate and offensive speech, there is a scarcity of datasets labeled with a finer granularity of target classes and specific targets. In this paper, we introduce THOS, a dataset of 8.3k tweets manually labeled with fine-grained annotations about the target of the message. We demonstrate that this dataset makes it feasible to train classifiers, based on Large Language Models, to perform classification at this level of granularity.