Enhancing Federated Quadruplet Learning: Stochastic Client Selection and Embedding Stability Analysis
For federated learning practitioners, FedQuad addresses representation misalignment due to non-IID data, but the improvements are incremental over existing methods.
FedQuad enhances federated learning by using quadruplet loss to improve representation alignment under data heterogeneity, achieving consistent accuracy gains over baselines on CIFAR-10, CIFAR-100, and Tiny-ImageNet.
Federated Learning (FL) enables decentralised model training across distributed clients without requiring data centralisation. However, the generalisation performance of the global model is usually degraded by data heterogeneity across clients, particularly under limited data availability and class imbalance. To address this challenge, we propose FedQuad, a novel method that explicitly enforces minimising intra-class representations while enabling inter-class splits across clients. By jointly minimising distances between positive pairs and maximising distances between negative pairs, the proposed approach mitigates representation misalignment introduced during model aggregation. We evaluate our method on CIFAR-10, CIFAR-100, and Tiny-ImageNet under diverse non-IID settings and varying numbers of clients, demonstrating consistent improvements over existing baselines. Additionally, we provide a comprehensive analysis of metric learning-based approaches in both centralised and federated environments, highlighting their effectiveness in alleviating representation collapse under heterogeneous data distributions.