Ensemble-based Uncertainty Quantification: Bayesian versus Credal Inference
This work addresses uncertainty quantification for machine learning practitioners, but it appears incremental as it compares existing methods without introducing a new paradigm.
The paper tackled the problem of quantifying aleatoric and epistemic uncertainty in ensemble-based methods, comparing Bayesian and credal inference approaches, and found that their effectiveness was evaluated empirically in classification with a reject option, though no concrete numbers were provided in the abstract.
The idea to distinguish and quantify two important types of uncertainty, often referred to as aleatoric and epistemic, has received increasing attention in machine learning research in the last couple of years. In this paper, we consider ensemble-based approaches to uncertainty quantification. Distinguishing between different types of uncertainty-aware learning algorithms, we specifically focus on Bayesian methods and approaches based on so-called credal sets, which naturally suggest themselves from an ensemble learning point of view. For both approaches, we address the question of how to quantify aleatoric and epistemic uncertainty. The effectiveness of corresponding measures is evaluated and compared in an empirical study on classification with a reject option.