FLEdge: Benchmarking Federated Machine Learning Applications in Edge Computing Systems
This work addresses the problem of benchmarking FL in edge systems for researchers and practitioners, but it is incremental as it complements existing FL benchmarks.
The paper tackles the challenge of evaluating federated learning (FL) applications in edge computing systems by proposing FLEdge, a benchmark that systematically assesses client capabilities, computational and communication bottlenecks, client behavior, and data security, finding that embedded hardware has significant memory bottlenecks leading to 4x longer processing times compared to data center GPUs.
Federated Learning (FL) has become a viable technique for realizing privacy-enhancing distributed deep learning on the network edge. Heterogeneous hardware, unreliable client devices, and energy constraints often characterize edge computing systems. In this paper, we propose FLEdge, which complements existing FL benchmarks by enabling a systematic evaluation of client capabilities. We focus on computational and communication bottlenecks, client behavior, and data security implications. Our experiments with models varying from 14K to 80M trainable parameters are carried out on dedicated hardware with emulated network characteristics and client behavior. We find that state-of-the-art embedded hardware has significant memory bottlenecks, leading to 4x longer processing times than on modern data center GPUs.