Towards Uncertainty Quantification in Generative Model Learning
This addresses a fundamental gap in evaluating generative models for researchers and practitioners, but it is incremental as it builds on existing precision-recall methods.
The paper tackles the problem of uncertainty quantification in generative models, which is understudied despite concerns about reliability, and proposes formalizing this issue with preliminary experiments showing that aggregated precision-recall curves can capture model approximation uncertainty on synthetic datasets.
While generative models have become increasingly prevalent across various domains, fundamental concerns regarding their reliability persist. A crucial yet understudied aspect of these models is the uncertainty quantification surrounding their distribution approximation capabilities. Current evaluation methodologies focus predominantly on measuring the closeness between the learned and the target distributions, neglecting the inherent uncertainty in these measurements. In this position paper, we formalize the problem of uncertainty quantification in generative model learning. We discuss potential research directions, including the use of ensemble-based precision-recall curves. Our preliminary experiments on synthetic datasets demonstrate the effectiveness of aggregated precision-recall curves in capturing model approximation uncertainty, enabling systematic comparison among different model architectures based on their uncertainty characteristics.