Smart Sampling: Helping from Friendly Neighbors for Decentralized Federated Learning
This work addresses communication efficiency and model accuracy challenges in DFL for privacy-preserving distributed learning, representing an incremental improvement over existing methods.
The paper tackles the problem of data heterogeneity in Decentralized Federated Learning (DFL) by introducing AFIND+, an algorithm that samples and aggregates helpful neighbors to improve model performance, demonstrating superior results over other sampling algorithms on real-world datasets.
Federated Learning (FL) is gaining widespread interest for its ability to share knowledge while preserving privacy and reducing communication costs. Unlike Centralized FL, Decentralized FL (DFL) employs a network architecture that eliminates the need for a central server, allowing direct communication among clients and leading to significant communication resource savings. However, due to data heterogeneity, not all neighboring nodes contribute to enhancing the local client's model performance. In this work, we introduce \textbf{\emph{AFIND+}}, a simple yet efficient algorithm for sampling and aggregating neighbors in DFL, with the aim of leveraging collaboration to improve clients' model performance. AFIND+ identifies helpful neighbors, adaptively adjusts the number of selected neighbors, and strategically aggregates the sampled neighbors' models based on their contributions. Numerical results on real-world datasets with diverse data partitions demonstrate that AFIND+ outperforms other sampling algorithms in DFL and is compatible with most existing DFL optimization algorithms.