Evaluating Aleatoric Uncertainty via Conditional Generative Models
This work addresses the need for reliable uncertainty estimation in machine learning applications, offering a method that is less restrictive than previous approaches, though it appears incremental as it builds on existing conditional generative models.
The paper tackled the problem of aleatoric uncertainty quantification by using conditional generative models to estimate conditional distributions without strong restrictions on data distribution or dimensionality, and demonstrated that their metrics provide correct measurements and train competitive models against benchmarks.
Aleatoric uncertainty quantification seeks for distributional knowledge of random responses, which is important for reliability analysis and robustness improvement in machine learning applications. Previous research on aleatoric uncertainty estimation mainly targets closed-formed conditional densities or variances, which requires strong restrictions on the data distribution or dimensionality. To overcome these restrictions, we study conditional generative models for aleatoric uncertainty estimation. We introduce two metrics to measure the discrepancy between two conditional distributions that suit these models. Both metrics can be easily and unbiasedly computed via Monte Carlo simulation of the conditional generative models, thus facilitating their evaluation and training. We demonstrate numerically how our metrics provide correct measurements of conditional distributional discrepancies and can be used to train conditional models competitive against existing benchmarks.