ModelNet40-E: An Uncertainty-Aware Benchmark for Point Cloud Classification
This provides a domain-specific benchmark for assessing uncertainty modeling in point cloud classification, which is incremental as it builds on existing benchmarks by adding noise and uncertainty features.
The authors tackled the problem of evaluating point cloud classification models under synthetic LiDAR-like noise by introducing ModelNet40-E, a benchmark with noise-corrupted data and uncertainty annotations, and found that Point Transformer v3 showed superior calibration with uncertainties aligned with measurement uncertainty.
We introduce ModelNet40-E, a new benchmark designed to assess the robustness and calibration of point cloud classification models under synthetic LiDAR-like noise. Unlike existing benchmarks, ModelNet40-E provides both noise-corrupted point clouds and point-wise uncertainty annotations via Gaussian noise parameters (σ, μ), enabling fine-grained evaluation of uncertainty modeling. We evaluate three popular models-PointNet, DGCNN, and Point Transformer v3-across multiple noise levels using classification accuracy, calibration metrics, and uncertainty-awareness. While all models degrade under increasing noise, Point Transformer v3 demonstrates superior calibration, with predicted uncertainties more closely aligned with the underlying measurement uncertainty.