YUVMultiNet: Real-time YUV multi-task CNN for autonomous driving
This work addresses the need for efficient, real-time perception in autonomous vehicles, though it is incremental as it builds on existing multi-task CNN methods with optimizations for specific hardware constraints.
The authors tackled the problem of real-time multi-task CNN inference for autonomous driving by proposing YUVMultiNet, a unified architecture with a shared encoder for detection and segmentation that runs at 25 FPS on 1280x800 resolution using a low-power automotive SoC.
In this paper, we propose a multi-task convolutional neural network (CNN) architecture optimized for a low power automotive grade SoC. We introduce a network based on a unified architecture where the encoder is shared among the two tasks namely detection and segmentation. The pro-posed network runs at 25FPS for 1280x800 resolution. We briefly discuss the methods used to optimize the network architecture such as using native YUV image directly, optimization of layers & feature maps and applying quantization. We also focus on memory bandwidth in our design as convolutions are data intensives and most SOCs are bandwidth bottlenecked. We then demonstrate the efficiency of our proposed network for a dedicated CNN accelerators presenting the key performance indicators (KPI) for the detection and segmentation tasks obtained from the hardware execution and the corresponding run-time.