Privacy Preserving Bayesian Federated Learning in Heterogeneous Settings
This addresses the need for uncertainty quantification and privacy in federated learning for clients with heterogeneous data and resources, offering a unified solution.
The paper tackled the problem of federated learning with heterogeneous clients, limited data, and privacy constraints by proposing a Bayesian framework that trains customized local models with functional priors, achieving superior performance and providing differential privacy guarantees.
In several practical applications of federated learning (FL), the clients are highly heterogeneous in terms of both their data and compute resources, and therefore enforcing the same model architecture for each client is very limiting. Moreover, the need for uncertainty quantification and data privacy constraints are often particularly amplified for clients that have limited local data. This paper presents a unified FL framework to simultaneously address all these constraints and concerns, based on training customized local Bayesian models that learn well even in the absence of large local datasets. A Bayesian framework provides a natural way of incorporating supervision in the form of prior distributions. We use priors in the functional (output) space of the networks to facilitate collaboration across heterogeneous clients. Moreover, formal differential privacy guarantees are provided for this framework. Experiments on standard FL datasets demonstrate that our approach outperforms strong baselines in both homogeneous and heterogeneous settings and under strict privacy constraints, while also providing characterizations of model uncertainties.