CyclicFL: A Cyclic Model Pre-Training Approach to Efficient Federated Learning
This addresses efficiency and accuracy issues in federated learning for AIoT applications, offering an incremental improvement over existing methods by enhancing initial model guidance without compromising privacy.
The paper tackles the problem of slow convergence and poor accuracy in federated learning, especially in non-IID scenarios, by proposing CyclicFL, a method that pre-trains initial models cyclically on selected devices without exposing local data, resulting in up to 14.11% higher classification accuracy and faster training.
Federated learning (FL) has been proposed to enable distributed learning on Artificial Intelligence Internet of Things (AIoT) devices with guarantees of high-level data privacy. Since random initial models in FL can easily result in unregulated Stochastic Gradient Descent (SGD) processes, existing FL methods greatly suffer from both slow convergence and poor accuracy, especially in non-IID scenarios. To address this problem, we propose a novel method named CyclicFL, which can quickly derive effective initial models to guide the SGD processes, thus improving the overall FL training performance. We formally analyze the significance of data consistency between the pre-training and training stages of CyclicFL, showing the limited Lipschitzness of loss for the pre-trained models by CyclicFL. Moreover, we systematically prove that our method can achieve faster convergence speed under various convexity assumptions. Unlike traditional centralized pre-training methods that require public proxy data, CyclicFL pre-trains initial models on selected AIoT devices cyclically without exposing their local data. Therefore, they can be easily integrated into any security-critical FL methods. Comprehensive experimental results show that CyclicFL can not only improve the maximum classification accuracy by up to $14.11\%$ but also significantly accelerate the overall FL training process.