Boosting 3D Object Detection via Object-Focused Image Fusion
This work addresses the challenge of accurately classifying detected objects in 3D object detection for applications like robotics and autonomous driving, presenting an incremental improvement over existing methods.
The paper tackles the problem of incomplete geometric structures and lack of semantic information in point clouds for 3D object detection by fusing object-level image information into point features, resulting in a large improvement of +2.1 mAP@0.25 and +2.3 mAP@0.5 on the SUN RGB-D dataset.
3D object detection has achieved remarkable progress by taking point clouds as the only input. However, point clouds often suffer from incomplete geometric structures and the lack of semantic information, which makes detectors hard to accurately classify detected objects. In this work, we focus on how to effectively utilize object-level information from images to boost the performance of point-based 3D detector. We present DeMF, a simple yet effective method to fuse image information into point features. Given a set of point features and image feature maps, DeMF adaptively aggregates image features by taking the projected 2D location of the 3D point as reference. We evaluate our method on the challenging SUN RGB-D dataset, improving state-of-the-art results by a large margin (+2.1 mAP@0.25 and +2.3mAP@0.5). Code is available at https://github.com/haoy945/DeMF.