A framework for benchmarking uncertainty in deep regression
This work provides a tool for researchers to benchmark uncertainty quantification methods in deep regression, though it is incremental as it builds on existing statistical concepts without introducing new algorithms.
The authors tackled the problem of evaluating uncertainty quantification in deep regression by proposing a flexible benchmarking framework based on linear combinations of nonlinear functions, and they demonstrated its application by comparing deep regression methods against a statistical reference method using coverage probabilities and uncertainty sizes.
We propose a framework for the assessment of uncertainty quantification in deep regression. The framework is based on regression problems where the regression function is a linear combination of nonlinear functions. Basically, any level of complexity can be realized through the choice of the nonlinear functions and the dimensionality of their domain. Results of an uncertainty quantification for deep regression are compared against those obtained by a statistical reference method. The reference method utilizes knowledge of the underlying nonlinear functions and is based on a Bayesian linear regression using a reference prior. Reliability of uncertainty quantification is assessed in terms of coverage probabilities, and accuracy through the size of calculated uncertainties. We illustrate the proposed framework by applying it to current approaches for uncertainty quantification in deep regression. The flexibility, together with the availability of a reference solution, makes the framework suitable for defining benchmark sets for uncertainty quantification.