Eliminating Cross-modal Conflicts in BEV Space for LiDAR-Camera 3D Object Detection
This addresses a specific bottleneck in multi-sensor 3D object detection for autonomous driving, representing an incremental improvement over existing methods.
The paper tackles the problem of cross-modal conflicts in LiDAR-camera 3D object detection by proposing the ECFusion method, which eliminates extrinsic and inherent conflicts in BEV space to improve multi-modal features, achieving state-of-the-art performance on the nuScenes dataset.
Recent 3D object detectors typically utilize multi-sensor data and unify multi-modal features in the shared bird's-eye view (BEV) representation space. However, our empirical findings indicate that previous methods have limitations in generating fusion BEV features free from cross-modal conflicts. These conflicts encompass extrinsic conflicts caused by BEV feature construction and inherent conflicts stemming from heterogeneous sensor signals. Therefore, we propose a novel Eliminating Conflicts Fusion (ECFusion) method to explicitly eliminate the extrinsic/inherent conflicts in BEV space and produce improved multi-modal BEV features. Specifically, we devise a Semantic-guided Flow-based Alignment (SFA) module to resolve extrinsic conflicts via unifying spatial distribution in BEV space before fusion. Moreover, we design a Dissolved Query Recovering (DQR) mechanism to remedy inherent conflicts by preserving objectness clues that are lost in the fusion BEV feature. In general, our method maximizes the effective information utilization of each modality and leverages inter-modal complementarity. Our method achieves state-of-the-art performance in the highly competitive nuScenes 3D object detection dataset. The code is released at https://github.com/fjhzhixi/ECFusion.