Decentralized Personalized Federated Learning based on a Conditional Sparse-to-Sparser Scheme
This addresses efficiency and data heterogeneity issues in decentralized federated learning, offering incremental improvements for applications like edge computing.
The paper tackles the high training and communication costs in Decentralized Federated Learning (DFL) by proposing DA-DPFL, a sparse-to-sparser training scheme that reduces energy costs by up to 5 times while improving test accuracy compared to DFL baselines.
Decentralized Federated Learning (DFL) has become popular due to its robustness and avoidance of centralized coordination. In this paradigm, clients actively engage in training by exchanging models with their networked neighbors. However, DFL introduces increased costs in terms of training and communication. Existing methods focus on minimizing communication often overlooking training efficiency and data heterogeneity. To address this gap, we propose a novel \textit{sparse-to-sparser} training scheme: DA-DPFL. DA-DPFL initializes with a subset of model parameters, which progressively reduces during training via \textit{dynamic aggregation} and leads to substantial energy savings while retaining adequate information during critical learning periods. Our experiments showcase that DA-DPFL substantially outperforms DFL baselines in test accuracy, while achieving up to $5$ times reduction in energy costs. We provide a theoretical analysis of DA-DPFL's convergence by solidifying its applicability in decentralized and personalized learning. The code is available at:https://github.com/EricLoong/da-dpfl