Personalized Cross-Silo Federated Learning on Non-IID Data
This addresses the problem of data heterogeneity for federated learning systems, offering an incremental improvement over existing methods.
The paper tackles the challenge of non-IID data in federated learning by proposing FedAMP, a method that uses federated attentive message passing to enable pairwise collaborations between similar clients, showing superior performance in experiments.
Non-IID data present a tough challenge for federated learning. In this paper, we explore a novel idea of facilitating pairwise collaborations between clients with similar data. We propose FedAMP, a new method employing federated attentive message passing to facilitate similar clients to collaborate more. We establish the convergence of FedAMP for both convex and non-convex models, and propose a heuristic method to further improve the performance of FedAMP when clients adopt deep neural networks as personalized models. Our extensive experiments on benchmark data sets demonstrate the superior performance of the proposed methods.