Layerwise Geo-Distributed Computing between Cloud and IoT
This addresses communication bottlenecks in distributed deep learning for IoT applications, though it appears incremental as it builds on existing deep belief networks.
The paper tackles the problem of inefficient geo-distributed computing between Cloud and IoT by proposing a k-degree layer-wise network architecture, which reduces communication cost and learning time compared to a state-of-the-art model on the M-distributed MNIST database.
In this paper, we propose a novel architecture for a deep learning system, named k-degree layer-wise network, to realize efficient geo-distributed computing between Cloud and Internet of Things (IoT). The geo-distributed computing extends Cloud to the geographical verge of the network in the neighbor of IoT. The basic ideas of the proposal include a k-degree constraint and a layer-wise constraint. The k-degree constraint is defined such that the degree of each vertex on the h-th layer is exactly k(h) to extend the existing deep belief networks and control the communication cost. The layer-wise constraint is defined such that the layer-wise degrees are monotonically decreasing in positive direction to gradually reduce the dimension of data. We prove the k-degree layer-wise network is sparse, while a typical deep neural network is dense. In an evaluation on the M-distributed MNIST database, the proposal is superior to a state-of-the-art model in terms of communication cost and learning time with scalability.