Cross-Space Synergy: A Unified Framework for Multimodal Emotion Recognition in Conversation
This work addresses multimodal emotion recognition for conversational AI, presenting an incremental improvement over existing methods.
The paper tackled the problem of multimodal emotion recognition in conversation by addressing issues with capturing cross-modal interactions and training instability, proposing a unified framework called Cross-Space Synergy that improved accuracy and stability on IEMOCAP and MELD datasets.
Multimodal Emotion Recognition in Conversation (MERC) aims to predict speakers' emotions by integrating textual, acoustic, and visual cues. Existing approaches either struggle to capture complex cross-modal interactions or experience gradient conflicts and unstable training when using deeper architectures. To address these issues, we propose Cross-Space Synergy (CSS), which couples a representation component with an optimization component. Synergistic Polynomial Fusion (SPF) serves the representation role, leveraging low-rank tensor factorization to efficiently capture high-order cross-modal interactions. Pareto Gradient Modulator (PGM) serves the optimization role, steering updates along Pareto-optimal directions across competing objectives to alleviate gradient conflicts and improve stability. Experiments show that CSS outperforms existing representative methods on IEMOCAP and MELD in both accuracy and training stability, demonstrating its effectiveness in complex multimodal scenarios.