Fully Decentralized, Scalable Gaussian Processes for Multi-Agent Federated Learning
This work addresses scalability and decentralization challenges in multi-agent systems, though it appears incremental as it builds on existing ADMM and consensus techniques.
The paper tackles the problem of training and predicting with Gaussian processes in multi-agent federated learning by proposing decentralized algorithms using ADMM and consensus methods, achieving scalable performance as demonstrated in numerical experiments on synthetic and real data.
In this paper, we propose decentralized and scalable algorithms for Gaussian process (GP) training and prediction in multi-agent systems. To decentralize the implementation of GP training optimization algorithms, we employ the alternating direction method of multipliers (ADMM). A closed-form solution of the decentralized proximal ADMM is provided for the case of GP hyper-parameter training with maximum likelihood estimation. Multiple aggregation techniques for GP prediction are decentralized with the use of iterative and consensus methods. In addition, we propose a covariance-based nearest neighbor selection strategy that enables a subset of agents to perform predictions. The efficacy of the proposed methods is illustrated with numerical experiments on synthetic and real data.