Autolabeling 3D Objects with Differentiable Rendering of SDF Shape Priors
This addresses the need for efficient 3D annotation in autonomous driving, though it is incremental as it builds on existing methods like differentiable rendering and shape priors.
The paper tackles the problem of automatically annotating 3D objects from 2D detectors and sparse LIDAR data by solving an ill-posed inverse problem using learned shape priors and optimization, resulting in accurate cuboid recovery on the KITTI3D dataset and enabling training of 3D vehicle detectors with state-of-the-art performance.
We present an automatic annotation pipeline to recover 9D cuboids and 3D shapes from pre-trained off-the-shelf 2D detectors and sparse LIDAR data. Our autolabeling method solves an ill-posed inverse problem by considering learned shape priors and optimizing geometric and physical parameters. To address this challenging problem, we apply a novel differentiable shape renderer to signed distance fields (SDF), leveraged together with normalized object coordinate spaces (NOCS). Initially trained on synthetic data to predict shape and coordinates, our method uses these predictions for projective and geometric alignment over real samples. Moreover, we also propose a curriculum learning strategy, iteratively retraining on samples of increasing difficulty in subsequent self-improving annotation rounds. Our experiments on the KITTI3D dataset show that we can recover a substantial amount of accurate cuboids, and that these autolabels can be used to train 3D vehicle detectors with state-of-the-art results.