Distributed Learning for Time-varying Networks: A Scalable Design
This work addresses the problem of adapting distributed learning to dynamic network topologies for wireless communication systems, representing an incremental improvement over existing frameworks like federated learning.
The paper tackles the challenge of distributed learning in time-varying wireless networks by proposing a scalable deep neural network design that exploits permutation equivalence and invariance, resulting in improved learning convergence and performance as verified by simulation results.
The wireless network is undergoing a trend from "onnection of things" to "connection of intelligence". With data spread over the communication networks and computing capability enhanced on the devices, distributed learning becomes a hot topic in both industrial and academic communities. Many frameworks, such as federated learning and federated distillation, have been proposed. However, few of them takes good care of obstacles such as the time-varying topology resulted by the characteristics of wireless networks. In this paper, we propose a distributed learning framework based on a scalable deep neural network (DNN) design. By exploiting the permutation equivalence and invariance properties of the learning tasks, the DNNs with different scales for different clients can be built up based on two basic parameter sub-matrices. Further, model aggregation can also be conducted based on these two sub-matrices to improve the learning convergence and performance. Finally, simulation results verify the benefits of the proposed framework by compared with some baselines.