FedBC: Calibrating Global and Local Models via Federated Learning Beyond Consensus
This addresses the challenge of model calibration in federated learning for distributed devices, though it appears incremental as it builds on existing optimization methods.
The paper tackles the problem of balancing global and local model performance in federated learning by proposing FedBC, a multi-criterion optimization framework that calibrates these models through a primal-dual method, achieving competitive accuracy across datasets like MNIST and CIFAR-10.
In this work, we quantitatively calibrate the performance of global and local models in federated learning through a multi-criterion optimization-based framework, which we cast as a constrained program. The objective of a device is its local objective, which it seeks to minimize while satisfying nonlinear constraints that quantify the proximity between the local and the global model. By considering the Lagrangian relaxation of this problem, we develop a novel primal-dual method called Federated Learning Beyond Consensus (\texttt{FedBC}). Theoretically, we establish that \texttt{FedBC} converges to a first-order stationary point at rates that matches the state of the art, up to an additional error term that depends on a tolerance parameter introduced to scalarize the multi-criterion formulation. Finally, we demonstrate that \texttt{FedBC} balances the global and local model test accuracy metrics across a suite of datasets (Synthetic, MNIST, CIFAR-10, Shakespeare), achieving competitive performance with state-of-the-art.