CLIP: Client-Side Invariant Pruning for Mitigating Stragglers in Secure Federated Learning
This addresses performance bottlenecks in secure federated learning for heterogeneous devices, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackled the problem of straggler clients slowing down secure federated learning due to computational and network bottlenecks, and introduced CLIP, a client-side invariant pruning technique that accelerated training by 13% to 34% with minimal accuracy impact ranging from a 1.3% improvement to a 2.6% reduction.
Secure federated learning (FL) preserves data privacy during distributed model training. However, deploying such frameworks across heterogeneous devices results in performance bottlenecks, due to straggler clients with limited computational or network capabilities, slowing training for all participating clients. This paper introduces the first straggler mitigation technique for secure aggregation with deep neural networks. We propose CLIP, a client-side invariant neuron pruning technique coupled with network-aware pruning, that addresses compute and network bottlenecks due to stragglers during training with minimal accuracy loss. Our technique accelerates secure FL training by 13% to 34% across multiple datasets (CIFAR10, Shakespeare, FEMNIST) with an accuracy impact of between 1.3% improvement to 2.6% reduction.