Peer-to-peer Federated Learning on Graphs
This work addresses decentralized learning for networks, offering a solution for scenarios where centralized coordination is impractical, though it appears incremental as it builds on existing federated and Bayesian approaches.
The paper tackles the problem of training machine learning models in a fully decentralized network by proposing a Bayesian-like distributed algorithm where nodes update beliefs using neighbor information, resulting in significant accuracy improvements compared to non-cooperative learning in experiments with linear regression and deep neural networks.
We consider the problem of training a machine learning model over a network of nodes in a fully decentralized framework. The nodes take a Bayesian-like approach via the introduction of a belief over the model parameter space. We propose a distributed learning algorithm in which nodes update their belief by aggregate information from their one-hop neighbors to learn a model that best fits the observations over the entire network. In addition, we also obtain sufficient conditions to ensure that the probability of error is small for every node in the network. We discuss approximations required for applying this algorithm to train Deep Neural Networks (DNNs). Experiments on training linear regression model and on training a DNN show that the proposed learning rule algorithm provides a significant improvement in the accuracy compared to the case where nodes learn without cooperation.