Why have a Unified Predictive Uncertainty? Disentangling it using Deep Split Ensembles
This work addresses uncertainty quantification for neural networks in critical domains like healthcare, offering a deployable solution with minimal changes, though it is incremental as it builds on existing unified uncertainty methods.
The authors tackled the problem of disentangling predictive uncertainty in neural networks by proposing deep split ensembles, a non-Bayesian method that clusters input features to estimate uncertainty per cluster, achieving well-calibrated models on benchmark datasets and revealing hidden modality-specific biases in an Alzheimer's dataset.
Understanding and quantifying uncertainty in black box Neural Networks (NNs) is critical when deployed in real-world settings such as healthcare. Recent works using Bayesian and non-Bayesian methods have shown how a unified predictive uncertainty can be modelled for NNs. Decomposing this uncertainty to disentangle the granular sources of heteroscedasticity in data provides rich information about its underlying causes. We propose a conceptually simple non-Bayesian approach, deep split ensemble, to disentangle the predictive uncertainties using a multivariate Gaussian mixture model. The NNs are trained with clusters of input features, for uncertainty estimates per cluster. We evaluate our approach on a series of benchmark regression datasets, while also comparing with unified uncertainty methods. Extensive analyses using dataset shits and empirical rule highlight our inherently well-calibrated models. Our work further demonstrates its applicability in a multi-modal setting using a benchmark Alzheimer's dataset and also shows how deep split ensembles can highlight hidden modality-specific biases. The minimal changes required to NNs and the training procedure, and the high flexibility to group features into clusters makes it readily deployable and useful. The source code is available at https://github.com/wazeerzulfikar/deep-split-ensembles