Estimating Uncertainty with Implicit Quantile Network
This provides a practical tool for applications like healthcare where knowing model uncertainty is critical, though it is incremental compared to existing methods.
The paper tackles uncertainty quantification in deep learning by proposing a simple alternative using an Implicit Quantile Network to model loss distributions, resulting in a 2x higher mean estimated loss for incorrect predictions and up to 10% accuracy improvement when removing high-uncertainty data.
Uncertainty quantification is an important part of many performance critical applications. This paper provides a simple alternative to existing approaches such as ensemble learning and bayesian neural networks. By directly modeling the loss distribution with an Implicit Quantile Network, we get an estimate of how uncertain the model is of its predictions. For experiments with MNIST and CIFAR datasets, the mean of the estimated loss distribution is 2x higher for incorrect predictions. When data with high estimated uncertainty is removed from the test dataset, the accuracy of the model goes up as much as 10%. This method is simple to implement while offering important information to applications where the user has to know when the model could be wrong (e.g. deep learning for healthcare).