Boosting Single-Frame 3D Object Detection by Simulating Multi-Frame Point Clouds
This work improves 3D object detection for autonomous driving by enhancing single-frame detectors without requiring multi-frame inputs at inference, though it is incremental as it builds on existing distillation and fusion techniques.
The paper tackles the problem of single-frame 3D object detection by training a detector to simulate features from multi-frame point clouds, achieving state-of-the-art performance on the Waymo test set with improved mAP and mAPH for all object classes and difficulty levels.
To boost a detector for single-frame 3D object detection, we present a new approach to train it to simulate features and responses following a detector trained on multi-frame point clouds. Our approach needs multi-frame point clouds only when training the single-frame detector, and once trained, it can detect objects with only single-frame point clouds as inputs during the inference. We design a novel Simulated Multi-Frame Single-Stage object Detector (SMF-SSD) framework to realize the approach: multi-view dense object fusion to densify ground-truth objects to generate a multi-frame point cloud; self-attention voxel distillation to facilitate one-to-many knowledge transfer from multi- to single-frame voxels; multi-scale BEV feature distillation to transfer knowledge in low-level spatial and high-level semantic BEV features; and adaptive response distillation to activate single-frame responses of high confidence and accurate localization. Experimental results on the Waymo test set show that our SMF-SSD consistently outperforms all state-of-the-art single-frame 3D object detectors for all object classes of difficulty levels 1 and 2 in terms of both mAP and mAPH.