OpenAD: Open-World Autonomous Driving Benchmark for 3D Object Detection
This work addresses the problem of limited adaptability to novel domains and corner cases in autonomous driving perception for researchers and developers, but it is incremental as it builds upon existing datasets and methods.
The paper tackles the lack of comprehensive open-world 3D perception benchmarks by introducing OpenAD, the first real open-world autonomous driving benchmark for 3D object detection, which includes 2000 scenarios from five datasets and shows that their proposed baseline and ensemble method improve performance, though specific numerical gains are not detailed in the abstract.
Open-world perception aims to develop a model adaptable to novel domains and various sensor configurations and can understand uncommon objects and corner cases. However, current research lacks sufficiently comprehensive open-world 3D perception benchmarks and robust generalizable methodologies. This paper introduces OpenAD, the first real open-world autonomous driving benchmark for 3D object detection. OpenAD is built upon a corner case discovery and annotation pipeline that integrates with a multimodal large language model (MLLM). The proposed pipeline annotates corner case objects in a unified format for five autonomous driving perception datasets with 2000 scenarios. In addition, we devise evaluation methodologies and evaluate various open-world and specialized 2D and 3D models. Moreover, we propose a vision-centric 3D open-world object detection baseline and further introduce an ensemble method by fusing general and specialized models to address the issue of lower precision in existing open-world methods for the OpenAD benchmark. We host an online challenge on EvalAI. Data, toolkit codes, and evaluation codes are available at https://github.com/VDIGPKU/OpenAD.