PromptMTopic: Unsupervised Multimodal Topic Modeling of Memes using Large Language Models
This addresses content moderation, social media analysis, and cultural studies by providing a method to analyze memes, though it is incremental as it builds on existing topic modeling and multimodal techniques.
The paper tackles unsupervised multimodal topic modeling of memes by proposing PromptMTopic, which leverages large language models to learn topics from text and visual elements, demonstrating superiority over state-of-the-art baselines on three real-world datasets.
The proliferation of social media has given rise to a new form of communication: memes. Memes are multimodal and often contain a combination of text and visual elements that convey meaning, humor, and cultural significance. While meme analysis has been an active area of research, little work has been done on unsupervised multimodal topic modeling of memes, which is important for content moderation, social media analysis, and cultural studies. We propose \textsf{PromptMTopic}, a novel multimodal prompt-based model designed to learn topics from both text and visual modalities by leveraging the language modeling capabilities of large language models. Our model effectively extracts and clusters topics learned from memes, considering the semantic interaction between the text and visual modalities. We evaluate our proposed model through extensive experiments on three real-world meme datasets, which demonstrate its superiority over state-of-the-art topic modeling baselines in learning descriptive topics in memes. Additionally, our qualitative analysis shows that \textsf{PromptMTopic} can identify meaningful and culturally relevant topics from memes. Our work contributes to the understanding of the topics and themes of memes, a crucial form of communication in today's society.\\ \red{\textbf{Disclaimer: This paper contains sensitive content that may be disturbing to some readers.}}