QUARC: Quaternion Multi-Modal Fusion Architecture For Hate Speech Classification
This work provides a more efficient method for hate speech detection from social media posts, which is beneficial for platform moderators and researchers in natural language processing.
The paper addresses the problem of hate speech classification using multi-modal data (text and image). It proposes a quaternion neural network-based model that achieves a 75% reduction in parameters while maintaining comparable performance to existing models.
Hate speech, quite common in the age of social media, at times harmless but can also cause mental trauma to someone or even riots in communities. Image of a religious symbol with derogatory comment or video of a man abusing a particular community, all become hate speech with its every modality (such as text, image, and audio) contributing towards it. Models based on a particular modality of hate speech post on social media are not useful, rather, we need models like multi-modal fusion models that consider both image and text while classifying hate speech. Text-image fusion models are heavily parameterized, hence we propose a quaternion neural network-based model having additional fusion components for each pair of modalities. The model is tested on the MMHS150K twitter dataset for hate speech classification. The model shows an almost 75% reduction in parameters and also benefits us in terms of storage space and training time while being at par in terms of performance as compared to its real counterpart.