Sparse Personalized Federated Learning
This work addresses efficiency and personalization issues in federated learning for distributed clients, but it is incremental as it builds on existing methods with sparse constraints.
The paper tackles challenges in federated learning, such as statistical diversity and communication overhead, by proposing FedMac, a sparse personalized federated learning scheme that improves performance on non-i.i.d. data and reduces computational and communication loads, achieving up to 99.37% accuracy on datasets like FMNIST.
Federated Learning (FL) is a collaborative machine learning technique to train a global model without obtaining clients' private data. The main challenges in FL are statistical diversity among clients, limited computing capability among clients' equipments, and the excessive communication overhead between the server and clients. To address these challenges, we propose a novel sparse personalized federated learning scheme via maximizing correlation (FedMac). By incorporating an approximated L1-norm and the correlation between client models and global model into standard FL loss function, the performance on statistical diversity data is improved and the communicational and computational loads required in the network are reduced compared with non-sparse FL. Convergence analysis shows that the sparse constraints in FedMac do not affect the convergence rate of the global model, and theoretical results show that FedMac can achieve good sparse personalization, which is better than the personalized methods based on L2-norm. Experimentally, we demonstrate the benefits of this sparse personalization architecture compared with the state-of-the-art personalization methods (e.g. FedMac respectively achieves 98.95%, 99.37%, 90.90%, 89.06% and 73.52% accuracy on the MNIST, FMNIST, CIFAR-100, Synthetic and CINIC-10 datasets under non-i.i.d. variants).