Labels Are Not Perfect: Inferring Spatial Uncertainty in Object Detection
This work tackles the problem of inaccurate and deterministic object labels for autonomous driving, which affects the training and evaluation of object detection algorithms, particularly for probabilistic object detection.
This paper addresses the issue of imperfect bounding box labels in object detection datasets by inferring spatial uncertainty from LiDAR point clouds using a generative model. They define a new probabilistic bounding box representation and propose Jaccard IoU (JIoU) as an evaluation metric that incorporates label uncertainty. The method improves detection accuracy when integrated into a loss function for training probabilistic object detectors.
The availability of many real-world driving datasets is a key reason behind the recent progress of object detection algorithms in autonomous driving. However, there exist ambiguity or even failures in object labels due to error-prone annotation process or sensor observation noise. Current public object detection datasets only provide deterministic object labels without considering their inherent uncertainty, as does the common training process or evaluation metrics for object detectors. As a result, an in-depth evaluation among different object detection methods remains challenging, and the training process of object detectors is sub-optimal, especially in probabilistic object detection. In this work, we infer the uncertainty in bounding box labels from LiDAR point clouds based on a generative model, and define a new representation of the probabilistic bounding box through a spatial uncertainty distribution. Comprehensive experiments show that the proposed model reflects complex environmental noises in LiDAR perception and the label quality. Furthermore, we propose Jaccard IoU (JIoU) as a new evaluation metric that extends IoU by incorporating label uncertainty. We conduct an in-depth comparison among several LiDAR-based object detectors using the JIoU metric. Finally, we incorporate the proposed label uncertainty in a loss function to train a probabilistic object detector and to improve its detection accuracy. We verify our proposed methods on two public datasets (KITTI, Waymo), as well as on simulation data. Code is released at https://bit.ly/2W534yo.