Dynamic Multimodal Sentiment Analysis: Leveraging Cross-Modal Attention for Enabled Classification
This work addresses sentiment analysis for applications like human-computer interaction, but it is incremental as it builds on existing multimodal methods with minor improvements.
This paper tackled the problem of improving sentiment classification by integrating text, audio, and visual data, finding that early stage fusion achieved 71.87% accuracy and multi-headed attention reached 72.39% on the CMU-MOSEI dataset.
This paper explores the development of a multimodal sentiment analysis model that integrates text, audio, and visual data to enhance sentiment classification. The goal is to improve emotion detection by capturing the complex interactions between these modalities, thereby enabling more accurate and nuanced sentiment interpretation. The study evaluates three feature fusion strategies -- late stage fusion, early stage fusion, and multi-headed attention -- within a transformer-based architecture. Experiments were conducted using the CMU-MOSEI dataset, which includes synchronized text, audio, and visual inputs labeled with sentiment scores. Results show that early stage fusion significantly outperforms late stage fusion, achieving an accuracy of 71.87\%, while the multi-headed attention approach offers marginal improvement, reaching 72.39\%. The findings suggest that integrating modalities early in the process enhances sentiment classification, while attention mechanisms may have limited impact within the current framework. Future work will focus on refining feature fusion techniques, incorporating temporal data, and exploring dynamic feature weighting to further improve model performance.