Federated Learning on Adaptively Weighted Nodes by Bilevel Optimization
This work addresses performance optimization in federated learning for distributed systems, but it appears incremental as it builds on existing federated learning and bilevel optimization frameworks.
The paper tackles the problem of improving federated learning performance by adaptively weighting nodes, formulating it as a bilevel optimization where inner federated learning with weighted nodes is optimized by outer weight tuning based on validation performance. The result includes a communication-efficient algorithm and theoretical analysis showing superiority over local training and static weight methods under certain assumptions.
We propose a federated learning method with weighted nodes in which the weights can be modified to optimize the model's performance on a separate validation set. The problem is formulated as a bilevel optimization where the inner problem is a federated learning problem with weighted nodes and the outer problem focuses on optimizing the weights based on the validation performance of the model returned from the inner problem. A communication-efficient federated optimization algorithm is designed to solve this bilevel optimization problem. Under an error-bound assumption, we analyze the generalization performance of the output model and identify scenarios when our method is in theory superior to training a model only locally and to federated learning with static and evenly distributed weights.