Collaborative Execution of Deep Neural Networks on Internet of Things Devices
This addresses the problem of limited compute resources and privacy concerns for IoT users, offering a domain-specific incremental improvement over existing edge optimizations.
The paper tackles the challenge of deploying compute-intensive deep neural networks on resource-constrained IoT devices by proposing a collaborative network approach that aggregates computing power for real-time, single-batch inferencing, achieving results such as enhanced processing pipelines demonstrated with models like AlexNet and VGG16 on Raspberry Pis.
With recent advancements in deep neural networks (DNNs), we are able to solve traditionally challenging problems. Since DNNs are compute intensive, consumers, to deploy a service, need to rely on expensive and scarce compute resources in the cloud. This approach, in addition to its dependability on high-quality network infrastructure and data centers, raises new privacy concerns. These challenges may limit DNN-based applications, so many researchers have tried optimize DNNs for local and in-edge execution. However, inadequate power and computing resources of edge devices along with small number of requests limits current optimizations applicability, such as batch processing. In this paper, we propose an approach that utilizes aggregated existing computing power of Internet of Things (IoT) devices surrounding an environment by creating a collaborative network. In this approach, IoT devices cooperate to conduct single-batch inferencing in real time. While exploiting several new model-parallelism methods and their distribution characteristics, our approach enhances the collaborative network by creating a balanced and distributed processing pipeline. We have illustrated our work using many Raspberry Pis with studying DNN models such as AlexNet, VGG16, Xception, and C3D.