Peer-to-Peer Deep Learning for Beyond-5G IoT
This addresses scalability and reliability issues for IoT applications like smart cities, offering a practical alternative to federated learning, though it is incremental in its approach.
The paper tackles the problem of scaling deep learning for IoT in beyond-5G environments by proposing P2PL, a peer-to-peer algorithm that eliminates the need for central servers, and shows it achieves the same test performance as federated and centralized training with up to 100 devices.
We present P2PL, a practical multi-device peer-to-peer deep learning algorithm that, unlike the federated learning paradigm, does not require coordination from edge servers or the cloud. This makes P2PL well-suited for the sheer scale of beyond-5G computing environments like smart cities that otherwise create range, latency, bandwidth, and single point of failure issues for federated approaches. P2PL introduces max norm synchronization to catalyze training, retains on-device deep model training to preserve privacy, and leverages local inter-device communication to implement distributed consensus. Each device iteratively alternates between two phases: 1) on-device learning and 2) peer-to-peer cooperation where they combine model parameters with nearby devices. We empirically show that all participating devices achieve the same test performance attained by federated and centralized training -- even with 100 devices and relaxed singly stochastic consensus weights. We extend these experimental results to settings with diverse network topologies, sparse and intermittent communication, and non-IID data distributions.