Global Update Guided Federated Learning
This addresses data imbalance issues in federated learning for privacy-preserving applications, representing an incremental improvement over existing methods.
The paper tackles the problem of unbalanced data distributions in federated learning, which compromises accuracy and convergence speed, by proposing FedGG with a model-cosine loss and adaptive loss weights, resulting in significant improvements in convergence accuracies and speeds compared to other advanced algorithms.
Federated learning protects data privacy and security by exchanging models instead of data. However, unbalanced data distributions among participating clients compromise the accuracy and convergence speed of federated learning algorithms. To alleviate this problem, unlike previous studies that limit the distance of updates for local models, we propose global-update-guided federated learning (FedGG), which introduces a model-cosine loss into local objective functions, so that local models can fit local data distributions under the guidance of update directions of global models. Furthermore, considering that the update direction of a global model is informative in the early stage of training, we propose adaptive loss weights based on the update distances of local models. Numerical simulations show that, compared with other advanced algorithms, FedGG has a significant improvement on model convergence accuracies and speeds. Additionally, compared with traditional fixed loss weights, adaptive loss weights enable our algorithm to be more stable and easier to implement in practice.