HateTinyLLM : Hate Speech Detection Using Tiny Large Language Models
This work addresses automated hate speech detection for social media platforms, but it is incremental as it applies existing fine-tuning methods to new models.
The authors tackled hate speech detection by fine-tuning tiny large language models (tinyLLMs) like TinyLlama-1.1B and phi-2, achieving over 80% accuracy and outperforming the pretrained mixtral-7b model.
Hate speech encompasses verbal, written, or behavioral communication that targets derogatory or discriminatory language against individuals or groups based on sensitive characteristics. Automated hate speech detection plays a crucial role in curbing its propagation, especially across social media platforms. Various methods, including recent advancements in deep learning, have been devised to address this challenge. In this study, we introduce HateTinyLLM, a novel framework based on fine-tuned decoder-only tiny large language models (tinyLLMs) for efficient hate speech detection. Our experimental findings demonstrate that the fine-tuned HateTinyLLM outperforms the pretrained mixtral-7b model by a significant margin. We explored various tiny LLMs, including PY007/TinyLlama-1.1B-step-50K-105b, Microsoft/phi-2, and facebook/opt-1.3b, and fine-tuned them using LoRA and adapter methods. Our observations indicate that all LoRA-based fine-tuned models achieved over 80\% accuracy.