LDRFusion: A LiDAR-Dominant multimodal refinement framework for 3D object detection
This addresses 3D object detection for autonomous vehicles, with incremental improvements in fusion methods.
The paper tackled the problem of noise from pseudo point clouds in LiDAR-Camera fusion for 3D object detection by proposing LDRFusion, a LiDAR-dominant two-stage refinement framework, resulting in strong performance on the KITTI dataset across categories and difficulty levels.
Existing LiDAR-Camera fusion methods have achieved strong results in 3D object detection. To address the sparsity of point clouds, previous approaches typically construct spatial pseudo point clouds via depth completion as auxiliary input and adopts a proposal-refinement framework to generate detection results. However, introducing pseudo points inevitably brings noise, potentially resulting in inaccurate predictions. Considering the differing roles and reliability levels of each modality, we propose LDRFusion, a novel Lidar-dominant two-stage refinement framework for multi-sensor fusion. The first stage soley relies on LiDAR to produce accurately localized proposals, followed by a second stage where pseudo point clouds are incorporated to detect challenging instances. The instance-level results from both stages are subsequently merged. To further enhance the representation of local structures in pseudo point clouds, we present a hierarchical pseudo point residual encoding module, which encodes neighborhood sets using both feature and positional residuals. Experiments on the KITTI dataset demonstrate that our framework consistently achieves strong performance across multiple categories and difficulty levels.