Efficient Computer Vision on Edge Devices with Pipeline-Parallel Hierarchical Neural Networks
This work addresses efficiency challenges for computer vision applications on edge devices, such as search-and-rescue and security, with incremental improvements over existing parallelization techniques.
The paper tackles the problem of inefficient computer vision on low-power edge devices by proposing a pipeline-parallel method for hierarchical neural networks, achieving 3.21X higher throughput, 68% less energy consumption, and 58% less memory compared to single-device approaches.
Computer vision on low-power edge devices enables applications including search-and-rescue and security. State-of-the-art computer vision algorithms, such as Deep Neural Networks (DNNs), are too large for inference on low-power edge devices. To improve efficiency, some existing approaches parallelize DNN inference across multiple edge devices. However, these techniques introduce significant communication and synchronization overheads or are unable to balance workloads across devices. This paper demonstrates that the hierarchical DNN architecture is well suited for parallel processing on multiple edge devices. We design a novel method that creates a parallel inference pipeline for computer vision problems that use hierarchical DNNs. The method balances loads across the collaborating devices and reduces communication costs to facilitate the processing of multiple video frames simultaneously with higher throughput. Our experiments consider a representative computer vision problem where image recognition is performed on each video frame, running on multiple Raspberry Pi 4Bs. With four collaborating low-power edge devices, our approach achieves 3.21X higher throughput, 68% less energy consumption per device per frame, and 58% decrease in memory when compared with existing single-device hierarchical DNNs.