Performance Measurement for Deep Bayesian Neural Network
This work addresses a gap in evaluating Bayesian neural networks, which is crucial for applications in life-threatening areas where uncertainty quantification is important, though it is incremental in focusing on measurement rather than novel methods.
The paper tackles the lack of specific performance metrics for deep Bayesian neural networks, proposing new criteria such as model calibration, data rejection ability, and uncertainty divergence to evaluate models beyond general measures like accuracy.
Deep Bayesian neural network has aroused a great attention in recent years since it combines the benefits of deep neural network and probability theory. Because of this, the network can make predictions and quantify the uncertainty of the predictions at the same time, which is important in many life-threatening areas. However, most of the recent researches are mainly focusing on making the Bayesian neural network easier to train, and proposing methods to estimate the uncertainty. I notice there are very few works that properly discuss the ways to measure the performance of the Bayesian neural network. Although accuracy and average uncertainty are commonly used for now, they are too general to provide any insight information about the model. In this paper, we would like to introduce more specific criteria and propose several metrics to measure the model performance from different perspectives, which include model calibration measurement, data rejection ability and uncertainty divergence for samples from the same and different distributions.