FedBEns: One-Shot Federated Learning based on Bayesian Ensemble
This work addresses communication efficiency in federated learning for distributed clients, representing an incremental improvement over existing methods.
The paper tackled the problem of One-Shot Federated Learning by proposing FedBEns, a Bayesian ensemble method that leverages multimodality in local loss functions, resulting in improved performance over baselines that use unimodal approximations.
One-Shot Federated Learning (FL) is a recent paradigm that enables multiple clients to cooperatively learn a global model in a single round of communication with a central server. In this paper, we analyze the One-Shot FL problem through the lens of Bayesian inference and propose FedBEns, an algorithm that leverages the inherent multimodality of local loss functions to find better global models. Our algorithm leverages a mixture of Laplace approximations for the clients' local posteriors, which the server then aggregates to infer the global model. We conduct extensive experiments on various datasets, demonstrating that the proposed method outperforms competing baselines that typically rely on unimodal approximations of the local losses.