Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles
This work addresses uncertainty quantification in regression for machine learning practitioners, offering an incremental improvement in methods for generating interpretable prediction intervals.
The paper tackles the problem of generating well-calibrated prediction intervals and point estimates in regression by proposing a multi-objective loss function and a split normal mixture aggregation method for neural network ensembles, resulting in improved calibration of uncertainty estimates.
Prediction intervals are a machine- and human-interpretable way to represent predictive uncertainty in a regression analysis. In this paper, we present a method for generating prediction intervals along with point estimates from an ensemble of neural networks. We propose a multi-objective loss function fusing quality measures related to prediction intervals and point estimates, and a penalty function, which enforces semantic integrity of the results and stabilizes the training process of the neural networks. The ensembled prediction intervals are aggregated as a split normal mixture accounting for possible multimodality and asymmetricity of the posterior predictive distribution, and resulting in prediction intervals that capture aleatoric and epistemic uncertainty. Our results show that both our quality-driven loss function and our aggregation method contribute to well-calibrated prediction intervals and point estimates.