Deep Equilibrium Models Meet Federated Learning
This addresses efficiency and scalability issues for federated learning systems with heterogeneous edge devices, but it is incremental as it adapts an existing model type to a new context.
The paper tackles the problem of communication overhead and device heterogeneity in Federated Learning by using Deep Equilibrium models, proposing a weighted average fusion rule, and shows promising initial experimental results.
In this study the problem of Federated Learning (FL) is explored under a new perspective by utilizing the Deep Equilibrium (DEQ) models instead of conventional deep learning networks. We claim that incorporating DEQ models into the federated learning framework naturally addresses several open problems in FL, such as the communication overhead due to the sharing large models and the ability to incorporate heterogeneous edge devices with significantly different computation capabilities. Additionally, a weighted average fusion rule is proposed at the server-side of the FL framework to account for the different qualities of models from heterogeneous edge devices. To the best of our knowledge, this study is the first to establish a connection between DEQ models and federated learning, contributing to the development of an efficient and effective FL framework. Finally, promising initial experimental results are presented, demonstrating the potential of this approach in addressing challenges of FL.