Manifold-Aware Perturbations for Constrained Generative Modeling
This addresses a mathematical limitation in generative modeling for scientific applications where data must satisfy equality constraints.
The paper tackles the problem of generative models struggling with equality-constrained distributions common in scientific domains, proposing a constraint-aware perturbation method that enables consistent data distribution recovery and stable sampling with diffusion models and normalizing flows.
Generative models have enjoyed widespread success in a variety of applications. However, they encounter inherent mathematical limitations in modeling distributions where samples are constrained by equalities, as is frequently the setting in scientific domains. In this work, we develop a computationally cheap, mathematically justified, and highly flexible distributional modification for combating known pitfalls in equality-constrained generative models. We propose perturbing the data distribution in a constraint-aware way such that the new distribution has support matching the ambient space dimension while still implicitly incorporating underlying manifold geometry. Through theoretical analyses and empirical evidence on several representative tasks, we illustrate that our approach consistently enables data distribution recovery and stable sampling with both diffusion models and normalizing flows.