Resource-Aware Heterogeneous Federated Learning using Neural Architecture Search
This work addresses the challenge of optimizing models in privacy-preserving, heterogeneous federated networks for edge computing, though it appears incremental by combining existing techniques like NAS and FL.
The paper tackles the problem of training AI models in federated learning with heterogeneous data and computational resources by proposing Resource-aware Federated Learning (RaFL), which uses neural architecture search to allocate specialized models to edge devices and achieves superior resource efficiency compared to state-of-the-art methods.
Federated Learning (FL) is extensively used to train AI/ML models in distributed and privacy-preserving settings. Participant edge devices in FL systems typically contain non-independent and identically distributed (Non-IID) private data and unevenly distributed computational resources. Preserving user data privacy while optimizing AI/ML models in a heterogeneous federated network requires us to address data and system/resource heterogeneity. To address these challenges, we propose Resource-aware Federated Learning (RaFL). RaFL allocates resource-aware specialized models to edge devices using Neural Architecture Search (NAS) and allows heterogeneous model architecture deployment by knowledge extraction and fusion. Combining NAS and FL enables on-demand customized model deployment for resource-diverse edge devices. Furthermore, we propose a multi-model architecture fusion scheme allowing the aggregation of the distributed learning results. Results demonstrate RaFL's superior resource efficiency compared to SoTA.