FedCache 2.0: Federated Edge Learning with Knowledge Caching and Dataset Distillation
This work addresses practical deployment issues in federated learning for edge devices, offering a solution for efficient and personalized training with limited communication, though it appears incremental as an extension of prior caching and distillation techniques.
FedCache 2.0 tackles challenges in Federated Edge Learning by combining dataset distillation and knowledge caching to enable personalized model training on edge devices, achieving at least a 28.6x improvement in communication efficiency and outperforming state-of-the-art methods across diverse datasets.
Federated Edge Learning (FEL) has emerged as a promising approach for enabling edge devices to collaboratively train machine learning models while preserving data privacy. Despite its advantages, practical FEL deployment faces significant challenges related to device constraints and device-server interactions, necessitating heterogeneous, user-adaptive model training with limited and uncertain communication. In this paper, we introduce FedCache 2.0, a novel personalized FEL architecture that simultaneously addresses these challenges. FedCache 2.0 incorporates the benefits of both dataset distillation and knowledge cache-driven federated learning by storing and organizing distilled data as knowledge in the server-side knowledge cache. Moreover, a device-centric cache sampling strategy is introduced to tailor transferred knowledge for individual devices within controlled communication bandwidth. Extensive experiments on five datasets covering image recognition, audio understanding, and mobile sensor data mining tasks demonstrate that (1) FedCache 2.0 significantly outperforms state-of-the-art methods regardless of model structures, data distributions, and modalities. (2) FedCache 2.0 can train splendid personalized on-device models with at least $\times$28.6 improvement in communication efficiency.