Learning to Validate Generative Models: a Goodness-of-Fit Approach
This addresses the need for scalable and interpretable validation methods for generative models in scientific workflows, particularly in high-energy physics, though it is incremental as it adapts an existing learning-based approach to new applications.
The paper tackles the problem of validating generative models in high-dimensional scientific settings by proposing the New Physics Learning Machine (NPLM), a goodness-of-fit testing approach, and demonstrates its performance on Gaussian mixtures and jet data from the Large Hadron Collider, showing it can diagnose sub-optimally modeled regions.
Generative models are increasingly central to scientific workflows, yet their systematic use and interpretation require a proper understanding of their limitations through rigorous validation. Classic approaches struggle with scalability, statistical power, or interpretability when applied to high-dimensional data, making it difficult to certify the reliability of these models in realistic, high-dimensional scientific settings. Here, we propose the use of the New Physics Learning Machine (NPLM), a learning based approach to goodness-of-fit testing inspired by the Neyman-Pearson construction, to test generative networks trained on high-dimensional scientific data. We demonstrate the performance of NPLM for validation in two benchmark cases: generative models trained on mixtures of Gaussian models with increasing dimensionality, and a public end-to-end generator for the Large Hadron Collider called FlashSim, trained on jet data, typical in the field of high-energy physics. We demonstrate that the NPLM can serve as a powerful validation method while also providing a means to diagnose sub-optimally modeled regions of the data.