Are we Missing Confidence in Pseudo-LiDAR Methods for Monocular 3D Object Detection?
This work addresses the problem of biased performance evaluation for Pseudo-LiDAR methods in monocular 3D object detection, which is significant for researchers and practitioners relying on the KITTI3D benchmark.
This paper reveals that the reported performance gains of Pseudo-LiDAR (PL) methods on the KITTI3D validation set are biased due to data overlap, making the test set the only reliable comparison. The authors then propose a novel deep architecture with a 3D confidence prediction module, achieving state-of-the-art performance on the KITTI3D benchmark.
Pseudo-LiDAR-based methods for monocular 3D object detection have received considerable attention in the community due to the performance gains exhibited on the KITTI3D benchmark, in particular on the commonly reported validation split. This generated a distorted impression about the superiority of Pseudo-LiDAR-based (PL-based) approaches over methods working with RGB images only. Our first contribution consists in rectifying this view by pointing out and showing experimentally that the validation results published by PL-based methods are substantially biased. The source of the bias resides in an overlap between the KITTI3D object detection validation set and the training/validation sets used to train depth predictors feeding PL-based methods. Surprisingly, the bias remains also after geographically removing the overlap. This leaves the test set as the only reliable set for comparison, where published PL-based methods do not excel. Our second contribution brings PL-based methods back up in the ranking with the design of a novel deep architecture which introduces a 3D confidence prediction module. We show that 3D confidence estimation techniques derived from RGB-only 3D detection approaches can be successfully integrated into our framework and, more importantly, that improved performance can be obtained with a newly designed 3D confidence measure, leading to state-of-the-art performance on the KITTI3D benchmark.