Cooperative Bayesian and variance networks disentangle aleatoric and epistemic uncertainties
This addresses the challenge of accurately modeling uncertainties in machine learning for applications requiring robust predictions, though it is incremental as it builds on existing variance and Bayesian network methods.
The paper tackles the problem of disentangling aleatoric and epistemic uncertainties in real-world data by proposing a cooperative training method combining a variance network with a Bayesian neural network, resulting in improved mean estimation and effective uncertainty separation across diverse datasets, including a custom heteroscedastic regression dataset.
Real-world data contains aleatoric uncertainty - irreducible noise arising from imperfect measurements or from incomplete knowledge about the data generation process. Mean variance estimation (MVE) networks can learn this type of uncertainty but require ad-hoc regularization strategies to avoid overfitting and are unable to predict epistemic uncertainty (model uncertainty). Conversely, Bayesian neural networks predict epistemic uncertainty but are notoriously difficult to train due to the approximate nature of Bayesian inference. We propose to cooperatively train a variance network with a Bayesian neural network and demonstrate that the resulting model disentangles aleatoric and epistemic uncertainties while improving the mean estimation. We demonstrate the effectiveness and scalability of this method across a diverse range of datasets, including a time-dependent heteroscedastic regression dataset we created where the aleatoric uncertainty is known. The proposed method is straightforward to implement, robust, and adaptable to various model architectures.