Unleash the Potential of Image Branch for Cross-modal 3D Object Detection
This work addresses the problem of improving 3D object detection for autonomous vehicles by better integrating image and LiDAR data, representing an incremental advancement in cross-modal methods.
The paper tackles the underutilization of image data in cross-modal 3D object detection by proposing UPIDet, which introduces a normalized local coordinate map estimation task and a point-to-pixel module to enhance LiDAR-based detection, achieving top rank in the cyclist class on the KITTI benchmark.
To achieve reliable and precise scene understanding, autonomous vehicles typically incorporate multiple sensing modalities to capitalize on their complementary attributes. However, existing cross-modal 3D detectors do not fully utilize the image domain information to address the bottleneck issues of the LiDAR-based detectors. This paper presents a new cross-modal 3D object detector, namely UPIDet, which aims to unleash the potential of the image branch from two aspects. First, UPIDet introduces a new 2D auxiliary task called normalized local coordinate map estimation. This approach enables the learning of local spatial-aware features from the image modality to supplement sparse point clouds. Second, we discover that the representational capability of the point cloud backbone can be enhanced through the gradients backpropagated from the training objectives of the image branch, utilizing a succinct and effective point-to-pixel module. Extensive experiments and ablation studies validate the effectiveness of our method. Notably, we achieved the top rank in the highly competitive cyclist class of the KITTI benchmark at the time of submission. The source code is available at https://github.com/Eaphan/UPIDet.