A Theoretical Framework for Energy-Aware Gradient Pruning in Federated Learning
This work addresses energy efficiency for edge devices in Federated Learning, representing an incremental improvement over existing gradient pruning methods.
The paper tackled the problem of energy-agnostic gradient pruning in Federated Learning by proposing Cost-Weighted Magnitude Pruning (CWMP), which accounts for hardware-level disparities, and demonstrated that CWMP establishes a superior performance-energy Pareto frontier compared to the Top-K baseline on a non-IID CIFAR-10 benchmark.
Federated Learning (FL) is constrained by the communication and energy limitations of decentralized edge devices. While gradient sparsification via Top-K magnitude pruning effectively reduces the communication payload, it remains inherently energy-agnostic. It assumes all parameter updates incur identical downstream transmission and memory-update costs, ignoring hardware realities. We formalize the pruning process as an energy-constrained projection problem that accounts for the hardware-level disparities between memory-intensive and compute-efficient operations during the post-backpropagation phase. We propose Cost-Weighted Magnitude Pruning (CWMP), a selection rule that prioritizes parameter updates based on their magnitude relative to their physical cost. We demonstrate that CWMP is the optimal greedy solution to this constrained projection and provide a probabilistic analysis of its global energy efficiency. Numerical results on a non-IID CIFAR-10 benchmark show that CWMP consistently establishes a superior performance-energy Pareto frontier compared to the Top-K baseline.