Efficient Hate Speech Detection: A Three-Layer LoRA-Tuned BERTweet Framework
It addresses the problem of making robust hate speech detection accessible in resource-constrained environments, representing an incremental improvement with specific efficiency gains.
This paper tackled the challenge of developing computationally efficient hate speech detection systems by proposing a three-layer framework with rule-based pre-filtering and a LoRA-tuned BERTweet model, achieving a 0.85 macro F1 score (94% of SOTA performance) while using a model 100x smaller and training in 2 hours on a single T4 GPU.
This paper addresses the critical challenge of developing computationally efficient hate speech detection systems that maintain competitive performance while being practical for real-time deployment. We propose a novel three-layer framework that combines rule-based pre-filtering with a parameter-efficient LoRA-tuned BERTweet model and continuous learning capabilities. Our approach achieves 0.85 macro F1 score - representing 94% of the performance of state-of-the-art large language models like SafePhi (Phi-4 based) while using a base model that is 100x smaller (134M vs 14B parameters). Compared to traditional BERT-based approaches with similar computational requirements, our method demonstrates superior performance through strategic dataset unification and optimized fine-tuning. The system requires only 1.87M trainable parameters (1.37% of full fine-tuning) and trains in approximately 2 hours on a single T4 GPU, making robust hate speech detection accessible in resource-constrained environments while maintaining competitive accuracy for real-world deployment.