Tackling Computational Heterogeneity in FL: A Few Theoretical Insights
This work tackles the problem of resource constraints and data variability in federated learning for edge devices, representing an incremental advancement.
The paper addresses computational heterogeneity in federated learning by introducing a novel aggregation framework, analyzing it theoretically and experimentally to improve model generalization.
The future of machine learning lies in moving data collection along with training to the edge. Federated Learning, for short FL, has been recently proposed to achieve this goal. The principle of this approach is to aggregate models learned over a large number of distributed clients, i.e., resource-constrained mobile devices that collect data from their environment, to obtain a new more general model. The latter is subsequently redistributed to clients for further training. A key feature that distinguishes federated learning from data-center-based distributed training is the inherent heterogeneity. In this work, we introduce and analyse a novel aggregation framework that allows for formalizing and tackling computational heterogeneity in federated optimization, in terms of both heterogeneous data and local updates. Proposed aggregation algorithms are extensively analyzed from a theoretical, and an experimental prospective.