What My Motion tells me about Your Pose: A Self-Supervised Monocular 3D Vehicle Detector
This work addresses the data-hungry and generalization issues in 3D vehicle detection for autonomous vehicles, offering a practical solution for leveraging unlabeled data from commercial fleets, though it is incremental as it builds on existing self-supervised and optimization techniques.
The paper tackles the problem of estimating 3D vehicle orientation and bounding boxes from monocular camera data for autonomous vehicles, which typically requires large labeled datasets and generalizes poorly. It introduces a self-supervised method using monocular visual odometry to fine-tune a pre-trained model, recovering up to 70% of the performance of fully supervised methods on the nuScenes dataset, and enables 3D detection without expensive labeled data.
The estimation of the orientation of an observed vehicle relative to an Autonomous Vehicle (AV) from monocular camera data is an important building block in estimating its 6 DoF pose. Current Deep Learning based solutions for placing a 3D bounding box around this observed vehicle are data hungry and do not generalize well. In this paper, we demonstrate the use of monocular visual odometry for the self-supervised fine-tuning of a model for orientation estimation pre-trained on a reference domain. Specifically, while transitioning from a virtual dataset (vKITTI) to nuScenes, we recover up to 70% of the performance of a fully supervised method. We subsequently demonstrate an optimization-based monocular 3D bounding box detector built on top of the self-supervised vehicle orientation estimator without the requirement of expensive labeled data. This allows 3D vehicle detection algorithms to be self-trained from large amounts of monocular camera data from existing commercial vehicle fleets.