From Black-box to White-box: Examining Confidence Calibration under different Conditions
This research addresses a critical issue for practitioners deploying object detection models in safety-critical applications, where reliable confidence estimates are crucial. It highlights how standard post-processing can negatively impact calibration.
This paper investigates confidence miscalibration in object detection models, specifically examining the impact of image location, box scale, and post-processing methods like non-maximum suppression (NMS). The study reveals that NMS can degrade initially well-calibrated predictions, leading to overconfident and miscalibrated models.
Confidence calibration is a major concern when applying artificial neural networks in safety-critical applications. Since most research in this area has focused on classification in the past, confidence calibration in the scope of object detection has gained more attention only recently. Based on previous work, we study the miscalibration of object detection models with respect to image location and box scale. Our main contribution is to additionally consider the impact of box selection methods like non-maximum suppression to calibration. We investigate the default intrinsic calibration of object detection models and how it is affected by these post-processing techniques. For this purpose, we distinguish between black-box calibration with non-maximum suppression and white-box calibration with raw network outputs. Our experiments reveal that post-processing highly affects confidence calibration. We show that non-maximum suppression has the potential to degrade initially well-calibrated predictions, leading to overconfident and thus miscalibrated models.