Decentralized federated learning of deep neural networks on non-iid data
This addresses the problem of efficient model learning in peer-to-peer systems with heterogeneous data distributions for decentralized AI applications, representing an incremental improvement.
The paper tackles decentralized federated learning with non-iid client data by proposing a method where clients detect and cooperate with peers having similar data distributions, achieving higher accuracies compared to strong baselines.
We tackle the non-convex problem of learning a personalized deep learning model in a decentralized setting. More specifically, we study decentralized federated learning, a peer-to-peer setting where data is distributed among many clients and where there is no central server to orchestrate the training. In real world scenarios, the data distributions are often heterogeneous between clients. Therefore, in this work we study the problem of how to efficiently learn a model in a peer-to-peer system with non-iid client data. We propose a method named Performance-Based Neighbor Selection (PENS) where clients with similar data distributions detect each other and cooperate by evaluating their training losses on each other's data to learn a model suitable for the local data distribution. Our experiments on benchmark datasets show that our proposed method is able to achieve higher accuracies as compared to strong baselines.