Federated Learning with Workload Reduction through Partial Training of Client Models and Entropy-Based Data Selection
This work addresses computational constraints for edge device users in federated learning, offering an incremental improvement by optimizing data selection and training efficiency.
The paper tackles the problem of high training workload on edge devices in federated learning by proposing FedFT-EDS, which combines partial model fine-tuning and entropy-based data selection to reduce data usage and training time. Experiments on CIFAR-10 and CIFAR-100 show it uses only 50% of user data while improving global model performance and achieving up to 3 times higher client learning efficiency with one third of the training time compared to baselines.
With the rapid expansion of edge devices, such as IoT devices, where crucial data needed for machine learning applications is generated, it becomes essential to promote their participation in privacy-preserving Federated Learning (FL) systems. The best way to achieve this desiderate is by reducing their training workload to match their constrained computational resources. While prior FL research has address the workload constrains by introducing lightweight models on the edge, limited attention has been given to optimizing on-device training efficiency through reducing the amount of data need during training. In this work, we propose FedFT-EDS, a novel approach that combines Fine-Tuning of partial client models with Entropy-based Data Selection to reduce training workloads on edge devices. By actively selecting the most informative local instances for learning, FedFT-EDS reduces training data significantly in FL and demonstrates that not all user data is equally beneficial for FL on all rounds. Our experiments on CIFAR-10 and CIFAR-100 show that FedFT-EDS uses only 50% user data while improving the global model performance compared to baseline methods, FedAvg and FedProx. Importantly, FedFT-EDS improves client learning efficiency by up to 3 times, using one third of training time on clients to achieve an equivalent performance to the baselines. This work highlights the importance of data selection in FL and presents a promising pathway to scalable and efficient Federate Learning.