Towards Domain Generalization for Multi-view 3D Object Detection in Bird-Eye-View
This addresses domain shift issues in camera-only 3D object detection for autonomous driving, offering a novel approach to improve robustness across different datasets, though it is incremental as it builds on existing MV3D-Det methods.
The paper tackles the problem of domain generalization for multi-view 3D object detection in Bird-Eye-View, where performance degrades when input images differ from training domains, and proposes DG-BEV to decouple depth estimation and use adversarial training, achieving reduced performance drop on unseen domains without harming source domain accuracy as shown in experiments on Waymo, nuScenes, and Lyft datasets.
Multi-view 3D object detection (MV3D-Det) in Bird-Eye-View (BEV) has drawn extensive attention due to its low cost and high efficiency. Although new algorithms for camera-only 3D object detection have been continuously proposed, most of them may risk drastic performance degradation when the domain of input images differs from that of training. In this paper, we first analyze the causes of the domain gap for the MV3D-Det task. Based on the covariate shift assumption, we find that the gap mainly attributes to the feature distribution of BEV, which is determined by the quality of both depth estimation and 2D image's feature representation. To acquire a robust depth prediction, we propose to decouple the depth estimation from the intrinsic parameters of the camera (i.e. the focal length) through converting the prediction of metric depth to that of scale-invariant depth and perform dynamic perspective augmentation to increase the diversity of the extrinsic parameters (i.e. the camera poses) by utilizing homography. Moreover, we modify the focal length values to create multiple pseudo-domains and construct an adversarial training loss to encourage the feature representation to be more domain-agnostic. Without bells and whistles, our approach, namely DG-BEV, successfully alleviates the performance drop on the unseen target domain without impairing the accuracy of the source domain. Extensive experiments on various public datasets, including Waymo, nuScenes, and Lyft, demonstrate the generalization and effectiveness of our approach. To the best of our knowledge, this is the first systematic study to explore a domain generalization method for MV3D-Det.