NCGR: Noise-Conditional Gated Rectification for Camera Extrinsic Perturbations in BEV 3D Object Detection
This addresses the robustness of camera-based BEV 3D detection to extrinsic perturbations, which is critical for real-world autonomous driving, but the improvement is specific to this domain and the method is an incremental enhancement over existing perturbation-handling techniques.
The paper proposes NCGR, a method to compensate for camera extrinsic perturbations in BEV 3D object detection by predicting 2D rectification offsets modulated by a camera-level gate, without explicit 6-DoF correction. On nuScenes with simulated perturbations, NCGR achieves 39.69% NDS in a five-camera dynamic stress test, outperforming BEVFormer (28.00%) and CAPE (33.23%), while maintaining clean-extrinsic performance comparable to BEVFormer.
Camera-based bird's-eye-view (BEV) 3D detection typically assumes accurate and fixed camera extrinsics. In detectors using spatial cross-attention (SCA), extrinsic perturbations displace the image-plane projections of BEV reference points, causing queries to sample features from incorrect regions and degrading detection performance. To address this failure mode, Noise-Conditional Gated Rectification (NCGR) is proposed to compensate for projection errors without explicitly estimating a full six-degree-of-freedom extrinsic correction. For each query-camera pair, a 2D rectification offset is predicted and modulated by a camera-level gate to rectify the base projection before native deformable sampling. During training, the perturbation-derived quantities used to construct the condition and gate are gradually replaced through scheduled interpolation by counterparts generated from an auxiliary scalar predicted from camera features. This transition enables blind inference without perturbation metadata. During training, a weight-shared clean-teacher/perturbed-student pair is used, and the rectification module is supervised by a BEV-consistency objective between the two branches. NCGR is evaluated on nuScenes with simulated dynamic and static extrinsic perturbations. In a five-camera dynamic stress test, NCGR achieves 39.69% NDS, compared with 28.00% for BEVFormer and 33.23% for CAPE. Under clean extrinsics, NCGR maintains performance comparable to that of BEVFormer.