M2SA: Multimodal and Multilingual Model for Sentiment Analysis of Tweets
This work addresses a gap in multimodal and multilingual sentiment analysis for tweets, but it is incremental as it builds on existing datasets and methods.
The paper tackled the lack of multimodal sentiment analysis datasets for multilingual tweets by curating a new multimodal dataset from an existing textual one, and found that using a sentiment-tuned large language model as a text encoder performed exceptionally well in baseline experiments.
In recent years, multimodal natural language processing, aimed at learning from diverse data types, has garnered significant attention. However, there needs to be more clarity when it comes to analysing multimodal tasks in multi-lingual contexts. While prior studies on sentiment analysis of tweets have predominantly focused on the English language, this paper addresses this gap by transforming an existing textual Twitter sentiment dataset into a multimodal format through a straightforward curation process. Our work opens up new avenues for sentiment-related research within the research community. Additionally, we conduct baseline experiments utilising this augmented dataset and report the findings. Notably, our evaluations reveal that when comparing unimodal and multimodal configurations, using a sentiment-tuned large language model as a text encoder performs exceptionally well.