Can Large Vision-Language Models Understand Multimodal Sarcasm?
This work addresses sarcasm detection and explanation in multimodal contexts, which is important for sentiment analysis and emotion-sensitive tasks, but it is incremental as it builds on existing LVLM capabilities.
The paper tackles the problem of multimodal sarcasm analysis by evaluating Large Vision-Language Models (LVLMs) and identifying limitations like insufficient visual understanding, then proposes a training-free framework that integrates object extraction and external knowledge to improve performance, with experimental results demonstrating its effectiveness.
Sarcasm is a complex linguistic phenomenon that involves a disparity between literal and intended meanings, making it challenging for sentiment analysis and other emotion-sensitive tasks. While traditional sarcasm detection methods primarily focus on text, recent approaches have incorporated multimodal information. However, the application of Large Visual Language Models (LVLMs) in Multimodal Sarcasm Analysis (MSA) remains underexplored. In this paper, we evaluate LVLMs in MSA tasks, specifically focusing on Multimodal Sarcasm Detection and Multimodal Sarcasm Explanation. Through comprehensive experiments, we identify key limitations, such as insufficient visual understanding and a lack of conceptual knowledge. To address these issues, we propose a training-free framework that integrates in-depth object extraction and external conceptual knowledge to improve the model's ability to interpret and explain sarcasm in multimodal contexts. The experimental results on multiple models show the effectiveness of our proposed framework. The code is available at https://github.com/cp-cp/LVLM-MSA.