Efficient Heterogeneous Video Segmentation at the Edge
This addresses real-time video segmentation for resource-limited edge platforms, representing an incremental improvement in optimization.
The paper tackles efficient video segmentation for edge devices by designing network models and optimizing heterogeneous data flows, achieving higher accuracy with quadrupled resolution, shorter latency, higher frame rate, and lower power consumption.
We introduce an efficient video segmentation system for resource-limited edge devices leveraging heterogeneous compute. Specifically, we design network models by searching across multiple dimensions of specifications for the neural architectures and operations on top of already light-weight backbones, targeting commercially available edge inference engines. We further analyze and optimize the heterogeneous data flows in our systems across the CPU, the GPU and the NPU. Our approach has empirically factored well into our real-time AR system, enabling remarkably higher accuracy with quadrupled effective resolutions, yet at much shorter end-to-end latency, much higher frame rate, and even lower power consumption on edge platforms.