LiFT: Lightweight, FPGA-tailored 3D object detection based on LiDAR data
It addresses the need for efficient, low-complexity 3D object detection for autonomous systems using FPGAs, though it is incremental as it builds on existing techniques like reparameterizable convolutions.
This paper tackles the problem of real-time 3D object detection from LiDAR data on FPGA platforms by developing LiFT, a lightweight, fully quantized algorithm that achieves an mAP of 51.84% and NDS of 61.01% on the NuScenes validation dataset with a computational cost of 20.73 GMACs.
This paper presents LiFT, a lightweight, fully quantized 3D object detection algorithm for LiDAR data, optimized for real-time inference on FPGA platforms. Through an in-depth analysis of FPGA-specific limitations, we identify a set of FPGA-induced constraints that shape the algorithm's design. These include a computational complexity limit of 30 GMACs (billion multiply-accumulate operations), INT8 quantization for weights and activations, 2D cell-based processing instead of 3D voxels, and minimal use of skip connections. To meet these constraints while maximizing performance, LiFT combines novel mechanisms with state-of-the-art techniques such as reparameterizable convolutions and fully sparse architecture. Key innovations include the Dual-bound Pillar Feature Net, which boosts performance without increasing complexity, and an efficient scheme for INT8 quantization of input features. With a computational cost of just 20.73 GMACs, LiFT stands out as one of the few algorithms targeting minimal-complexity 3D object detection. Among comparable methods, LiFT ranks first, achieving an mAP of 51.84% and an NDS of 61.01% on the challenging NuScenes validation dataset. The code will be available at https://github.com/vision-agh/lift.