Bayesian Federated Learning with Hamiltonian Monte Carlo: Algorithm and Theory
It addresses uncertainty quantification in federated learning for distributed systems, but is incremental as it builds on prior Bayesian federated learning methods.
This paper tackles the problem of parameter estimation and uncertainty quantification in federated learning by introducing FA-HMC, a Bayesian algorithm with Hamiltonian Monte Carlo, and establishes rigorous convergence guarantees on non-iid data, showing it outperforms existing methods like FA-LD.
This work introduces a novel and efficient Bayesian federated learning algorithm, namely, the Federated Averaging stochastic Hamiltonian Monte Carlo (FA-HMC), for parameter estimation and uncertainty quantification. We establish rigorous convergence guarantees of FA-HMC on non-iid distributed data sets, under the strong convexity and Hessian smoothness assumptions. Our analysis investigates the effects of parameter space dimension, noise on gradients and momentum, and the frequency of communication (between the central node and local nodes) on the convergence and communication costs of FA-HMC. Beyond that, we establish the tightness of our analysis by showing that the convergence rate cannot be improved even for continuous FA-HMC process. Moreover, extensive empirical studies demonstrate that FA-HMC outperforms the existing Federated Averaging-Langevin Monte Carlo (FA-LD) algorithm.