On Calibrating Semantic Segmentation Models: Analyses and An Algorithm
This addresses the issue of over-confidence in semantic segmentation for computer vision applications, offering an incremental improvement over prior calibration methods.
The paper tackles the problem of confidence calibration in semantic segmentation models, identifying key factors like misprediction as a major cause of miscalibration and proposing a selective scaling method that outperforms existing approaches in experiments across various benchmarks.
We study the problem of semantic segmentation calibration. Lots of solutions have been proposed to approach model miscalibration of confidence in image classification. However, to date, confidence calibration research on semantic segmentation is still limited. We provide a systematic study on the calibration of semantic segmentation models and propose a simple yet effective approach. First, we find that model capacity, crop size, multi-scale testing, and prediction correctness have impact on calibration. Among them, prediction correctness, especially misprediction, is more important to miscalibration due to over-confidence. Next, we propose a simple, unifying, and effective approach, namely selective scaling, by separating correct/incorrect prediction for scaling and more focusing on misprediction logit smoothing. Then, we study popular existing calibration methods and compare them with selective scaling on semantic segmentation calibration. We conduct extensive experiments with a variety of benchmarks on both in-domain and domain-shift calibration and show that selective scaling consistently outperforms other methods.