Evaluating the Effectiveness of Natural Language Inference for Hate Speech Detection in Languages with Limited Labeled Data
This work addresses the problem of expanding hate speech detection to low-resource languages for researchers and practitioners, but it is incremental as it builds on existing NLI methods.
The paper tackled hate speech detection in languages with limited labeled data by testing natural language inference (NLI) models in zero- and few-shot settings, finding that NLI fine-tuning improves performance over direct fine-tuning but often underperforms compared to intermediate fine-tuning on English data, except when English data mismatches the test domain.
Most research on hate speech detection has focused on English where a sizeable amount of labeled training data is available. However, to expand hate speech detection into more languages, approaches that require minimal training data are needed. In this paper, we test whether natural language inference (NLI) models which perform well in zero- and few-shot settings can benefit hate speech detection performance in scenarios where only a limited amount of labeled data is available in the target language. Our evaluation on five languages demonstrates large performance improvements of NLI fine-tuning over direct fine-tuning in the target language. However, the effectiveness of previous work that proposed intermediate fine-tuning on English data is hard to match. Only in settings where the English training data does not match the test domain, can our customised NLI-formulation outperform intermediate fine-tuning on English. Based on our extensive experiments, we propose a set of recommendations for hate speech detection in languages where minimal labeled training data is available.