Target-oriented Multimodal Sentiment Classification with Counterfactual-enhanced Debiasing
This work addresses biases in multimodal sentiment analysis for specific targets, offering an incremental improvement over existing methods.
The paper tackles the problem of target-oriented multimodal sentiment classification by addressing dataset biases that cause spurious correlations between text features and labels, resulting in improved classification accuracy with state-of-the-art performance on benchmark datasets.
Target-oriented multimodal sentiment classification seeks to predict sentiment polarity for specific targets from image-text pairs. While existing works achieve competitive performance, they often over-rely on textual content and fail to consider dataset biases, in particular word-level contextual biases. This leads to spurious correlations between text features and output labels, impairing classification accuracy. In this paper, we introduce a novel counterfactual-enhanced debiasing framework to reduce such spurious correlations. Our framework incorporates a counterfactual data augmentation strategy that minimally alters sentiment-related causal features, generating detail-matched image-text samples to guide the model's attention toward content tied to sentiment. Furthermore, for learning robust features from counterfactual data and prompting model decisions, we introduce an adaptive debiasing contrastive learning mechanism, which effectively mitigates the influence of biased words. Experimental results on several benchmark datasets show that our proposed method outperforms state-of-the-art baselines.