Reliable Probability Intervals For Classification Using Inductive Venn Predictors Based on Distance Learning
This work addresses the need for reliable confidence measures in autonomous systems using deep learning, particularly for high-dimensional data like images and IoT applications, representing an incremental improvement over existing frameworks.
The paper tackles the problem of unknown prediction confidence in deep neural networks by using Inductive Venn Predictors with distance metric learning to compute reliable probability intervals for classification. The result shows improved accuracy and calibration in image classification and IoT botnet detection, with computational efficiency enabling real-time use.
Deep neural networks are frequently used by autonomous systems for their ability to learn complex, non-linear data patterns and make accurate predictions in dynamic environments. However, their use as black boxes introduces risks as the confidence in each prediction is unknown. Different frameworks have been proposed to compute accurate confidence measures along with the predictions but at the same time introduce a number of limitations like execution time overhead or inability to be used with high-dimensional data. In this paper, we use the Inductive Venn Predictors framework for computing probability intervals regarding the correctness of each prediction in real-time. We propose taxonomies based on distance metric learning to compute informative probability intervals in applications involving high-dimensional inputs. Empirical evaluation on image classification and botnet attacks detection in Internet-of-Things (IoT) applications demonstrates improved accuracy and calibration. The proposed method is computationally efficient, and therefore, can be used in real-time.