Beyond Classification: Definition and Density-based Estimation of Calibration in Object Detection
This work addresses the calibration of deep neural networks in safety-critical object detection applications, representing an incremental advancement by adapting classification concepts to this domain.
The paper tackles the problem of defining and estimating calibration error for object detection, proposing a consistent and differentiable estimator using kernel density estimation, which outperforms competing methods while maintaining similar detection performance.
Despite their impressive predictive performance in various computer vision tasks, deep neural networks (DNNs) tend to make overly confident predictions, which hinders their widespread use in safety-critical applications. While there have been recent attempts to calibrate DNNs, most of these efforts have primarily been focused on classification tasks, thus neglecting DNN-based object detectors. Although several recent works addressed calibration for object detection and proposed differentiable penalties, none of them are consistent estimators of established concepts in calibration. In this work, we tackle the challenge of defining and estimating calibration error specifically for this task. In particular, we adapt the definition of classification calibration error to handle the nuances associated with object detection, and predictions in structured output spaces more generally. Furthermore, we propose a consistent and differentiable estimator of the detection calibration error, utilizing kernel density estimation. Our experiments demonstrate the effectiveness of our estimator against competing train-time and post-hoc calibration methods, while maintaining similar detection performance.