Impact of Stickers on Multimodal Sentiment and Intent in Social Media: A New Task, Dataset and Baseline
This work addresses a gap in multimodal analysis for social media users, focusing on the impact of stickers, but it is incremental as it builds on existing sentiment and intent recognition methods with a new dataset and model.
The authors tackled the problem of analyzing sentiment and intent in social media chats involving stickers by proposing a new task (MSAIRS), creating a multimodal dataset with Chinese chat records, and developing a joint model (MMSAIR) that outperforms traditional models and advanced MLLMs, demonstrating improved recognition accuracy through mutual reinforcement.
Stickers are increasingly used in social media to express sentiment and intent. Despite their significant impact on sentiment analysis and intent recognition, little research has been conducted in this area. To address this gap, we propose a new task: \textbf{M}ultimodal chat \textbf{S}entiment \textbf{A}nalysis and \textbf{I}ntent \textbf{R}ecognition involving \textbf{S}tickers (MSAIRS). Additionally, we introduce a novel multimodal dataset containing Chinese chat records and stickers excerpted from several mainstream social media platforms. Our dataset includes paired data with the same text but different stickers, the same sticker but different contexts, and various stickers consisting of the same images with different texts, allowing us to better understand the impact of stickers on chat sentiment and intent. We also propose an effective multimodal joint model, MMSAIR, featuring differential vector construction and cascaded attention mechanisms for enhanced multimodal fusion. Our experiments demonstrate the necessity and effectiveness of jointly modeling sentiment and intent, as they mutually reinforce each other's recognition accuracy. MMSAIR significantly outperforms traditional models and advanced MLLMs, demonstrating the challenge and uniqueness of sticker interpretation in social media. Our dataset and code are available on https://github.com/FakerBoom/MSAIRS-Dataset.