Explainable Multimodal Sentiment Analysis on Bengali Memes
This work addresses the problem of understanding sentiment in Bengali memes for researchers and practitioners in natural language processing, but it is incremental as it builds on existing multimodal methods with a focus on a specific language.
The paper tackled sentiment analysis on Bengali memes, a low-resource language area, by using a multimodal approach with ResNet50 and BanglishBERT, achieving a weighted F1-score of 0.71 and applying explainable AI techniques for model interpretation.
Memes have become a distinctive and effective form of communication in the digital era, attracting online communities and cutting across cultural barriers. Even though memes are frequently linked with humor, they have an amazing capacity to convey a wide range of emotions, including happiness, sarcasm, frustration, and more. Understanding and interpreting the sentiment underlying memes has become crucial in the age of information. Previous research has explored text-based, image-based, and multimodal approaches, leading to the development of models like CAPSAN and PromptHate for detecting various meme categories. However, the study of low-resource languages like Bengali memes remains scarce, with limited availability of publicly accessible datasets. A recent contribution includes the introduction of the MemoSen dataset. However, the achieved accuracy is notably low, and the dataset suffers from imbalanced distribution. In this study, we employed a multimodal approach using ResNet50 and BanglishBERT and achieved a satisfactory result of 0.71 weighted F1-score, performed comparison with unimodal approaches, and interpreted behaviors of the models using explainable artificial intelligence (XAI) techniques.