Gradient-Free Generation for Hard-Constrained Systems
This addresses the problem of constrained generative modeling for scientific and engineering applications where physical laws must be strictly respected, representing a novel method for a known bottleneck.
The paper tackles the problem of generating samples that satisfy hard constraints in scientific applications where gradient information is sparse or expensive, by introducing a zero-shot framework called ECI sampling that adapts pre-trained flow-matching models without gradient computations or fine-tuning. The result shows that ECI-guided generation strictly adheres to constraints, outperforms baselines in zero-shot tasks, and achieves competitive results in regression tasks.
Generative models that satisfy hard constraints are critical in many scientific and engineering applications, where physical laws or system requirements must be strictly respected. Many existing constrained generative models, especially those developed for computer vision, rely heavily on gradient information, which is often sparse or computationally expensive in some fields, e.g., partial differential equations (PDEs). In this work, we introduce a novel framework for adapting pre-trained, unconstrained flow-matching models to satisfy constraints exactly in a zero-shot manner without requiring expensive gradient computations or fine-tuning. Our framework, ECI sampling, alternates between extrapolation (E), correction (C), and interpolation (I) stages during each iterative sampling step of flow matching sampling to ensure accurate integration of constraint information while preserving the validity of the generation. We demonstrate the effectiveness of our approach across various PDE systems, showing that ECI-guided generation strictly adheres to physical constraints and accurately captures complex distribution shifts induced by these constraints. Empirical results demonstrate that our framework consistently outperforms baseline approaches in various zero-shot constrained generation tasks and also achieves competitive results in the regression tasks without additional fine-tuning.