On the Calibration of Uncertainty Estimation in LiDAR-based Semantic Segmentation
This work addresses reliability issues in autonomous driving perception, though it is incremental as it builds on existing calibration methods with a focus on class imbalance.
The paper tackles the problem of confidence calibration for underrepresented classes in LiDAR-based semantic segmentation, proposing a metric to measure calibration quality per class and using it to evaluate uncertainty estimation methods and detect label errors in datasets.
The confidence calibration of deep learning-based perception models plays a crucial role in their reliability. Especially in the context of autonomous driving, downstream tasks like prediction and planning depend on accurate confidence estimates. In point-wise multiclass classification tasks like sematic segmentation the model has to deal with heavy class imbalances. Due to their underrepresentation, the confidence calibration of classes with smaller instances is challenging but essential, not only for safety reasons. We propose a metric to measure the confidence calibration quality of a semantic segmentation model with respect to individual classes. It is calculated by computing sparsification curves for each class based on the uncertainty estimates. We use the classification calibration metric to evaluate uncertainty estimation methods with respect to their confidence calibration of underrepresented classes. We furthermore suggest a double use for the method to automatically find label problems to improve the quality of hand- or auto-annotated datasets.