UniPAD: A Universal Pre-training Paradigm for Autonomous Driving
This addresses the need for better 3D scene understanding in autonomous driving, though it is an incremental improvement over existing methods.
The paper tackles the problem of feature learning for autonomous driving by introducing UniPAD, a self-supervised pre-training paradigm using 3D volumetric differentiable rendering, which improves lidar-, camera-, and lidar-camera-based baselines by 9.1, 7.7, and 6.9 NDS respectively and achieves state-of-the-art results of 73.2 NDS for 3D object detection and 79.4 mIoU for 3D semantic segmentation on nuScenes.
In the context of autonomous driving, the significance of effective feature learning is widely acknowledged. While conventional 3D self-supervised pre-training methods have shown widespread success, most methods follow the ideas originally designed for 2D images. In this paper, we present UniPAD, a novel self-supervised learning paradigm applying 3D volumetric differentiable rendering. UniPAD implicitly encodes 3D space, facilitating the reconstruction of continuous 3D shape structures and the intricate appearance characteristics of their 2D projections. The flexibility of our method enables seamless integration into both 2D and 3D frameworks, enabling a more holistic comprehension of the scenes. We manifest the feasibility and effectiveness of UniPAD by conducting extensive experiments on various downstream 3D tasks. Our method significantly improves lidar-, camera-, and lidar-camera-based baseline by 9.1, 7.7, and 6.9 NDS, respectively. Notably, our pre-training pipeline achieves 73.2 NDS for 3D object detection and 79.4 mIoU for 3D semantic segmentation on the nuScenes validation set, achieving state-of-the-art results in comparison with previous methods. The code will be available at https://github.com/Nightmare-n/UniPAD.