Connecting Federated ADMM to Bayes
This work addresses federated learning by bridging two distinct fields, offering incremental improvements through combination.
The paper tackles the problem of connecting federated learning approaches based on ADMM and Variational Bayes, showing that dual variables in ADMM emerge from VB parameters and deriving new ADMM variants with flexible covariances and functional regularization, validated through numerical experiments for improved performance.
We provide new connections between two distinct federated learning approaches based on (i) ADMM and (ii) Variational Bayes (VB), and propose new variants by combining their complementary strengths. Specifically, we show that the dual variables in ADMM naturally emerge through the 'site' parameters used in VB with isotropic Gaussian covariances. Using this, we derive two versions of ADMM from VB that use flexible covariances and functional regularisation, respectively. Through numerical experiments, we validate the improvements obtained in performance. The work shows connection between two fields that are believed to be fundamentally different and combines them to improve federated learning.