BLUE at Memotion 2.0 2022: You have my Image, my Text and my Transformer
This work addresses meme understanding for internet culture analysis, but it is incremental as it applies existing methods to a new dataset.
The paper tackled automatic meme classification for sentiment, humor, offensive, sarcasm, and motivation levels, using text-only BERT and a multi-modal transformer, achieving first, second, and third places in three shared tasks with the highest average score.
Memes are prevalent on the internet and continue to grow and evolve alongside our culture. An automatic understanding of memes propagating on the internet can shed light on the general sentiment and cultural attitudes of people. In this work, we present team BLUE's solution for the second edition of the MEMOTION shared task. We showcase two approaches for meme classification (i.e. sentiment, humour, offensive, sarcasm and motivation levels) using a text-only method using BERT, and a Multi-Modal-Multi-Task transformer network that operates on both the meme image and its caption to output the final scores. In both approaches, we leverage state-of-the-art pretrained models for text (BERT, Sentence Transformer) and image processing (EfficientNetV4, CLIP). Through our efforts, we obtain first place in task A, second place in task B and third place in task C. In addition, our team obtained the highest average score for all three tasks.