Automating 3D Dataset Generation with Neural Radiance Fields
This addresses the bottleneck of dataset scarcity for 3D detection in applications like robotics and augmented reality, offering an incremental improvement in automation.
The paper tackles the problem of expensive and limited 3D dataset creation for training detection models by proposing an automated pipeline using Neural Radiance Fields to generate high-quality 3D models for arbitrary objects, enabling synthetic dataset generation that achieves strong performance in 3D pose estimation tasks.
3D detection is a critical task to understand spatial characteristics of the environment and is used in a variety of applications including robotics, augmented reality, and image retrieval. Training performant detection models require diverse, precisely annotated, and large scale datasets that involve complex and expensive creation processes. Hence, there are only few public 3D datasets that are additionally limited in their range of classes. In this work, we propose a pipeline for automatic generation of 3D datasets for arbitrary objects. By utilizing the universal 3D representation and rendering capabilities of Radiance Fields, our pipeline generates high quality 3D models for arbitrary objects. These 3D models serve as input for a synthetic dataset generator. Our pipeline is fast, easy to use and has a high degree of automation. Our experiments demonstrate, that 3D pose estimation networks, trained with our generated datasets, archive strong performance in typical application scenarios.