Latent Feature-based Data Splits to Improve Generalisation Evaluation: A Hate Speech Detection Case Study
This work addresses the need for more robust evaluation methods in hate speech detection to prevent overestimation of model benefits, though it is incremental as it builds on existing datasets and models.
The authors tackled the problem of overfitting in hate speech detection models by creating new train-test splits based on clustering of hidden representations, revealing that models catastrophically fail on latent space blind spots with performance drops up to 40% in some cases.
With the ever-growing presence of social media platforms comes the increased spread of harmful content and the need for robust hate speech detection systems. Such systems easily overfit to specific targets and keywords, and evaluating them without considering distribution shifts that might occur between train and test data overestimates their benefit. We challenge hate speech models via new train-test splits of existing datasets that rely on the clustering of models' hidden representations. We present two split variants (Subset-Sum-Split and Closest-Split) that, when applied to two datasets using four pretrained models, reveal how models catastrophically fail on blind spots in the latent space. This result generalises when developing a split with one model and evaluating it on another. Our analysis suggests that there is no clear surface-level property of the data split that correlates with the decreased performance, which underscores that task difficulty is not always humanly interpretable. We recommend incorporating latent feature-based splits in model development and release two splits via the GenBench benchmark.