Human-in-the-Loop Hate Speech Classification in a Multilingual Context
This addresses the problem of maintaining robust hate speech detection post-deployment for multilingual online platforms, though it is incremental as it builds on existing BERT methods.
The paper tackled hate speech classification in multilingual contexts by developing a human-in-the-loop BERT-based pipeline, achieving an F1 score of 80.5 and outperforming the best existing multilingual classifier by 5.8 points in German and 3.6 points in French.
The shift of public debate to the digital sphere has been accompanied by a rise in online hate speech. While many promising approaches for hate speech classification have been proposed, studies often focus only on a single language, usually English, and do not address three key concerns: post-deployment performance, classifier maintenance and infrastructural limitations. In this paper, we introduce a new human-in-the-loop BERT-based hate speech classification pipeline and trace its development from initial data collection and annotation all the way to post-deployment. Our classifier, trained using data from our original corpus of over 422k examples, is specifically developed for the inherently multilingual setting of Switzerland and outperforms with its F1 score of 80.5 the currently best-performing BERT-based multilingual classifier by 5.8 F1 points in German and 3.6 F1 points in French. Our systematic evaluations over a 12-month period further highlight the vital importance of continuous, human-in-the-loop classifier maintenance to ensure robust hate speech classification post-deployment.