Dense Voxel Fusion for 3D Object Detection
This addresses the challenge of effectively combining camera and LiDAR data for more accurate 3D object detection in autonomous driving, though it is incremental as it builds on existing voxel-based methods.
The paper tackles the problem of underperforming fusion methods for 3D object detection in autonomous vehicles by proposing Dense Voxel Fusion (DVF), which generates multi-scale dense voxel features and uses projected ground truth labels, resulting in ranking 3rd on the KITTI benchmark and improving performance on the Waymo dataset.
Camera and LiDAR sensor modalities provide complementary appearance and geometric information useful for detecting 3D objects for autonomous vehicle applications. However, current end-to-end fusion methods are challenging to train and underperform state-of-the-art LiDAR-only detectors. Sequential fusion methods suffer from a limited number of pixel and point correspondences due to point cloud sparsity, or their performance is strictly capped by the detections of one of the modalities. Our proposed solution, Dense Voxel Fusion (DVF) is a sequential fusion method that generates multi-scale dense voxel feature representations, improving expressiveness in low point density regions. To enhance multi-modal learning, we train directly with projected ground truth 3D bounding box labels, avoiding noisy, detector-specific 2D predictions. Both DVF and the multi-modal training approach can be applied to any voxel-based LiDAR backbone. DVF ranks 3rd among published fusion methods on KITTI 3D car detection benchmark without introducing additional trainable parameters, nor requiring stereo images or dense depth labels. In addition, DVF significantly improves 3D vehicle detection performance of voxel-based methods on the Waymo Open Dataset.