Predict-Project-Renoise: Sampling Diffusion Models under Hard Constraints
This addresses the problem of ensuring physical accuracy in neural emulators for scientific applications, representing a novel method for a known bottleneck rather than an incremental improvement.
The paper tackled the problem of diffusion models failing to guarantee physical accuracy or constraint satisfaction in scientific applications by introducing a constrained sampling framework that enforces hard constraints during generation. The result was the Predict-Project-Renoise algorithm, which reduced constraint violations by over an order of magnitude and improved sample consistency in experiments on 2D distributions, PDEs, and global weather forecasting.
Neural emulators based on diffusion models show promise for scientific applications, but vanilla models cannot guarantee physical accuracy or constraint satisfaction. We address this by introducing a constrained sampling framework that enforces hard constraints, such as physical laws or observational consistency, at generation time. Our approach defines a constrained forward process that diffuses only over the feasible set of constraint-satisfying samples, inducing constrained marginal distributions. To reverse this, we propose Predict-Project-Renoise (PPR), an iterative algorithm that samples from the constrained marginals by alternating between denoising predictions, projecting onto the feasible set, and renoising. Experiments on 2D distributions, PDEs, and global weather forecasting demonstrate that PPR reduces constraint violations by over an order of magnitude while improving sample consistency and better matching the true constrained distribution compared to baselines.