Server Averaging for Federated Learning
This work addresses efficiency challenges in federated learning, which is crucial for privacy-preserving distributed training, but it appears incremental as it builds on existing methods like FedAvg.
The paper tackles the slow convergence and high computation costs in federated learning by proposing a server averaging algorithm that periodically averages previous global models, resulting in faster convergence to target accuracy and reduced client-level computation costs compared to FedAvg.
Federated learning allows distributed devices to collectively train a model without sharing or disclosing the local dataset with a central server. The global model is optimized by training and averaging the model parameters of all local participants. However, the improved privacy of federated learning also introduces challenges including higher computation and communication costs. In particular, federated learning converges slower than centralized training. We propose the server averaging algorithm to accelerate convergence. Sever averaging constructs the shared global model by periodically averaging a set of previous global models. Our experiments indicate that server averaging not only converges faster, to a target accuracy, than federated averaging (FedAvg), but also reduces the computation costs on the client-level through epoch decay.