Sparse Decentralized Federated Learning
This work addresses communication and computational bottlenecks for distributed nodes in federated learning, though it appears incremental as it builds on existing DFL methods with sparsity and privacy enhancements.
The paper tackles efficiency, stability, and trustworthiness challenges in Decentralized Federated Learning by introducing Sparse DFL with a novel algorithm called CEPS, which uses sparsity constraints and one-bit compressive sensing to significantly improve communication efficiency while integrating differential privacy for privacy preservation.
Decentralized Federated Learning (DFL) enables collaborative model training without a central server but faces challenges in efficiency, stability, and trustworthiness due to communication and computational limitations among distributed nodes. To address these critical issues, we introduce a sparsity constraint on the shared model, leading to Sparse DFL (SDFL), and propose a novel algorithm, CEPS. The sparsity constraint facilitates the use of one-bit compressive sensing to transmit one-bit information between partially selected neighbour nodes at specific steps, thereby significantly improving communication efficiency. Moreover, we integrate differential privacy into the algorithm to ensure privacy preservation and bolster the trustworthiness of the learning process. Furthermore, CEPS is underpinned by theoretical guarantees regarding both convergence and privacy. Numerical experiments validate the effectiveness of the proposed algorithm in improving communication and computation efficiency while maintaining a high level of trustworthiness.