Label-Free Model Failure Detection for Lidar-based Point Cloud Segmentation
This addresses model reliability in autonomous driving by enabling failure detection without expensive labeled data, though it is incremental as it builds on existing anomaly detection techniques.
The paper tackles the problem of detecting failures in lidar-based point cloud segmentation models for autonomous vehicles by using unlabeled data, introducing a method with supervised and self-supervised streams and presenting the LidarCODA dataset for evaluation.
Autonomous vehicles drive millions of miles on the road each year. Under such circumstances, deployed machine learning models are prone to failure both in seemingly normal situations and in the presence of outliers. However, in the training phase, they are only evaluated on small validation and test sets, which are unable to reveal model failures due to their limited scenario coverage. While it is difficult and expensive to acquire large and representative labeled datasets for evaluation, large-scale unlabeled datasets are typically available. In this work, we introduce label-free model failure detection for lidar-based point cloud segmentation, taking advantage of the abundance of unlabeled data available. We leverage different data characteristics by training a supervised and self-supervised stream for the same task to detect failure modes. We perform a large-scale qualitative analysis and present LidarCODA, the first publicly available dataset with labeled anomalies in real-world lidar data, for an extensive quantitative analysis.