Implementation of a perception system for autonomous vehicles using a detection-segmentation network in SoC FPGA
This work addresses the need for real-time, energy-efficient obstacle recognition in autonomous vehicles, though it is incremental as it applies an existing network to a new hardware platform.
The paper tackled the problem of developing an efficient perception system for autonomous vehicles by implementing a MultiTaskV3 detection-segmentation network on an AMD Xilinx Kria KV260 SoC FPGA, achieving over 97% mAP for detection and above 90% mIoU for segmentation while consuming only 5 watts on average.
Perception and control systems for autonomous vehicles are an active area of scientific and industrial research. These solutions should be characterised by high efficiency in recognising obstacles and other environmental elements in different road conditions, real-time capability, and energy efficiency. Achieving such functionality requires an appropriate algorithm and a suitable computing platform. In this paper, we have used the MultiTaskV3 detection-segmentation network as the basis for a perception system that can perform both functionalities within a single architecture. It was appropriately trained, quantised, and implemented on the AMD Xilinx Kria KV260 Vision AI embedded platform. By using this device, it was possible to parallelise and accelerate the computations. Furthermore, the whole system consumes relatively little power compared to a CPU-based implementation (an average of 5 watts, compared to the minimum of 55 watts for weaker CPUs, and the small size (119mm x 140mm x 36mm) of the platform allows it to be used in devices where the amount of space available is limited. It also achieves an accuracy higher than 97% of the mAP (mean average precision) for object detection and above 90% of the mIoU (mean intersection over union) for image segmentation. The article also details the design of the Mecanum wheel vehicle, which was used to test the proposed solution in a mock-up city.