Deep Gaussian Mixture Ensembles
This provides a novel probabilistic method for uncertainty quantification in deep learning, addressing a key challenge for applications requiring reliable predictions.
This paper tackles the problem of accurately quantifying epistemic and aleatoric uncertainty in deep learning by introducing deep Gaussian mixture ensembles (DGMEs), which outperform state-of-the-art models in handling complex predictive densities.
This work introduces a novel probabilistic deep learning technique called deep Gaussian mixture ensembles (DGMEs), which enables accurate quantification of both epistemic and aleatoric uncertainty. By assuming the data generating process follows that of a Gaussian mixture, DGMEs are capable of approximating complex probability distributions, such as heavy-tailed or multimodal distributions. Our contributions include the derivation of an expectation-maximization (EM) algorithm used for learning the model parameters, which results in an upper-bound on the log-likelihood of training data over that of standard deep ensembles. Additionally, the proposed EM training procedure allows for learning of mixture weights, which is not commonly done in ensembles. Our experimental results demonstrate that DGMEs outperform state-of-the-art uncertainty quantifying deep learning models in handling complex predictive densities.