Softly Constrained Denoisers for Diffusion Models
This addresses a common issue in scientific applications where constraint formulation is challenging, offering a method to enforce constraints without biasing the model away from the true data distribution.
The paper tackles the problem of diffusion models failing to produce constraint-compliant samples in scientific applications, proposing softly constrained denoisers that improve compliance while maintaining flexibility to handle misspecified constraints.
Diffusion models struggle to produce samples that respect constraints, a common requirement in scientific applications. Recent approaches have introduced regularization terms in the loss or guidance methods during sampling to enforce such constraints, but they bias the generative model away from the true data distribution. This is a problem when the constraint is misspecified, which is a common issue in scientific applications where constraint formulation is challenging. We propose to integrate guidance-inspired adjustments to the denoiser, instead of the loss or sampling loop. This achieves a soft inductive bias towards constraint-compliant samples. We show that these softly constrained denoisers exploit constraint knowledge to improve compliance over standard denoisers, while maintaining enough flexibility to deviate from it in case of misspecification with observed data.