FedTune: Automatic Tuning of Federated Learning Hyper-Parameters from System Perspective
This work addresses the system efficiency challenge for federated learning practitioners by automating hyper-parameter tuning, though it appears incremental as it builds on existing FL methods.
The paper tackled the problem of manually selecting federated learning hyper-parameters, which burdens practitioners due to varying application preferences, by proposing FedTune, an automatic tuning algorithm that achieved 8.48%-26.75% improvement over fixed hyper-parameters across different datasets.
Federated learning (FL) hyper-parameters significantly affect the training overheads in terms of computation time, transmission time, computation load, and transmission load. However, the current practice of manually selecting FL hyper-parameters puts a high burden on FL practitioners since various applications prefer different training preferences. In this paper, we propose FedTune, an automatic FL hyper-parameter tuning algorithm tailored to applications' diverse system requirements of FL training. FedTune is lightweight and flexible, achieving 8.48%-26.75% improvement for different datasets compared to fixed FL hyper-parameters.