FedUAF: Uncertainty-Aware Fusion with Reliability-Guided Aggregation for Multimodal Federated Sentiment Analysis
This work addresses robust multimodal sentiment analysis for federated learning applications, representing an incremental improvement with novel method integration.
The paper tackles challenges in multimodal sentiment analysis under federated learning, such as missing modalities and unreliable clients, by proposing FedUAF, which uses uncertainty-aware fusion and reliability-guided aggregation, achieving consistent outperformance over state-of-the-art baselines on CMU-MOSI and CMU-MOSEI datasets.
Multimodal sentiment analysis in federated learning environments faces significant challenges due to missing modalities, heterogeneous data distributions, and unreliable client updates. Existing federated approaches often struggle to maintain robust performance under these practical conditions. In this paper, we propose FedUAF, a unified multimodal federated learning framework that addresses these challenges through uncertainty-aware fusion and reliability-guided aggregation. FedUAF explicitly models modality-level uncertainty during local training and leverages client reliability to guide global aggregation, enabling effective learning under incomplete and noisy multimodal data. Extensive experiments on CMU-MOSI and CMU-MOSEI demonstrate that FedUAF consistently outperforms state-of-the-art federated baselines across various missing-modality patterns and Non-IID settings. Moreover, FedUAF exhibits superior robustness against noisy clients, highlighting its potential for real-world multimodal federated applications.