Aug3D-RPN: Improving Monocular 3D Object Detection by Synthetic Images with Virtual Depth
This work addresses performance bottlenecks in autonomous driving and robotics by improving monocular 3D detection without adding inference-time computational cost, though it is incremental as it builds on existing geometry-based models.
The paper tackles the problem of limited performance in monocular 3D object detection due to lack of accurate depth information, proposing a rendering module to synthesize images with virtual depths and an auxiliary depth estimation module during training, resulting in leading accuracy on the KITTI benchmark.
Current geometry-based monocular 3D object detection models can efficiently detect objects by leveraging perspective geometry, but their performance is limited due to the absence of accurate depth information. Though this issue can be alleviated in a depth-based model where a depth estimation module is plugged to predict depth information before 3D box reasoning, the introduction of such module dramatically reduces the detection speed. Instead of training a costly depth estimator, we propose a rendering module to augment the training data by synthesizing images with virtual-depths. The rendering module takes as input the RGB image and its corresponding sparse depth image, outputs a variety of photo-realistic synthetic images, from which the detection model can learn more discriminative features to adapt to the depth changes of the objects. Besides, we introduce an auxiliary module to improve the detection model by jointly optimizing it through a depth estimation task. Both modules are working in the training time and no extra computation will be introduced to the detection model. Experiments show that by working with our proposed modules, a geometry-based model can represent the leading accuracy on the KITTI 3D detection benchmark.