Communication-Efficient and Robust Multi-Modal Federated Learning via Latent-Space Consensus
This work addresses communication efficiency and robustness in multi-modal federated learning for distributed devices, but it appears incremental as it builds on existing FL methods with specific optimizations.
The paper tackled the problem of multi-modal federated learning with heterogeneous client modalities and architectures by introducing CoMFed, a framework using learnable projection matrices and latent-space regularization, which achieved competitive accuracy on human activity recognition benchmarks with minimal overhead.
Federated learning (FL) enables collaborative model training across distributed devices without sharing raw data, but applying FL to multi-modal settings introduces significant challenges. Clients typically possess heterogeneous modalities and model architectures, making it difficult to align feature spaces efficiently while preserving privacy and minimizing communication costs. To address this, we introduce CoMFed, a Communication-Efficient Multi-Modal Federated Learning framework that uses learnable projection matrices to generate compressed latent representations. A latent-space regularizer aligns these representations across clients, improving cross-modal consistency and robustness to outliers. Experiments on human activity recognition benchmarks show that CoMFed achieves competitive accuracy with minimal overhead.