NYCU-TWO at Memotion 3: Good Foundation, Good Teacher, then you have Good Meme Analysis
This work addresses the problem of multi-modal meme analysis for social media applications, presenting an incremental improvement over existing methods.
The paper tackled the Memotion 3.0 Shared Task for classifying emotion and intensity in memes by proposing a framework using CLIP for feature extraction, a Cooperative Teaching Model for sentiment analysis, and a Cascaded Emotion Classifier for emotion classification, achieving 2nd place in two tasks and 4th in another with weighted F1-scores of 0.342, 0.784, and 0.535.
This paper presents a robust solution to the Memotion 3.0 Shared Task. The goal of this task is to classify the emotion and the corresponding intensity expressed by memes, which are usually in the form of images with short captions on social media. Understanding the multi-modal features of the given memes will be the key to solving the task. In this work, we use CLIP to extract aligned image-text features and propose a novel meme sentiment analysis framework, consisting of a Cooperative Teaching Model (CTM) for Task A and a Cascaded Emotion Classifier (CEC) for Tasks B&C. CTM is based on the idea of knowledge distillation, and can better predict the sentiment of a given meme in Task A; CEC can leverage the emotion intensity suggestion from the prediction of Task C to classify the emotion more precisely in Task B. Experiments show that we achieved the 2nd place ranking for both Task A and Task B and the 4th place ranking for Task C, with weighted F1-scores of 0.342, 0.784, and 0.535 respectively. The results show the robustness and effectiveness of our framework. Our code is released at github.