Calibrating Bayesian Generative Machine Learning for Bayesiamplification
This work addresses uncertainty calibration for generative models in particle physics, which is an incremental improvement for enhancing reliability in simulation and inference tasks.
The paper tackles the problem of ill-defined uncertainty quantification in Bayesian generative machine learning models by proposing a calibration scheme, and demonstrates that well-calibrated uncertainties can estimate equivalent truth samples and indicate data amplification for smooth distribution features.
Recently, combinations of generative and Bayesian machine learning have been introduced in particle physics for both fast detector simulation and inference tasks. These neural networks aim to quantify the uncertainty on the generated distribution originating from limited training statistics. The interpretation of a distribution-wide uncertainty however remains ill-defined. We show a clear scheme for quantifying the calibration of Bayesian generative machine learning models. For a Continuous Normalizing Flow applied to a low-dimensional toy example, we evaluate the calibration of Bayesian uncertainties from either a mean-field Gaussian weight posterior, or Monte Carlo sampling network weights, to gauge their behaviour on unsteady distribution edges. Well calibrated uncertainties can then be used to roughly estimate the number of uncorrelated truth samples that are equivalent to the generated sample and clearly indicate data amplification for smooth features of the distribution.