PNCS:Power-Norm Cosine Similarity for Diverse Client Selection in Federated Learning
This addresses data heterogeneity challenges in federated learning for privacy-preserving distributed systems, representing an incremental improvement.
The paper tackles the problem of gradient correlation in federated learning under data heterogeneity by proposing Power-Norm Cosine Similarity (PNCS) for client selection, resulting in improved convergence speed and accuracy over state-of-the-art methods in experiments with a VGG16 model.
Federated Learning (FL) has emerged as a powerful paradigm for leveraging diverse datasets from multiple sources while preserving data privacy by avoiding centralized storage. However, many existing approaches fail to account for the intricate gradient correlations between remote clients, a limitation that becomes especially problematic in data heterogeneity scenarios. In this work, we propose a novel FL framework utilizing Power-Norm Cosine Similarity (PNCS) to improve client selection for model aggregation. By capturing higher-order gradient moments, PNCS addresses non-IID data challenges, enhancing convergence speed and accuracy. Additionally, we introduce a simple algorithm ensuring diverse client selection through a selection history queue. Experiments with a VGG16 model across varied data partitions demonstrate consistent improvements over state-of-the-art methods.