Constrained Discrete Diffusion
This addresses the need for controllable and safe sequence generation in applications such as text and molecule design, offering a training-free alternative to autoregressive models.
The paper tackled the problem of enforcing sequence-level constraints in generative models by introducing Constrained Discrete Diffusion (CDD), which integrates differentiable constraint optimization into the diffusion process, resulting in zero constraint violations across tasks like toxicity-controlled text generation and property-constrained molecule design.
Discrete diffusion models are a class of generative models that construct sequences by progressively denoising samples from a categorical noise distribution. Beyond their rapidly growing ability to generate coherent natural language, these models present a new and important opportunity to enforce sequence-level constraints, a capability that current autoregressive models cannot natively provide. This paper capitalizes on this opportunity by introducing Constrained Discrete Diffusion (CDD), a novel integration of differentiable constraint optimization within the diffusion process to ensure adherence to constraints, logic rules, or safety requirements for generated sequences. Unlike conventional text generators that often rely on post-hoc filtering or model retraining for controllable generation, CDD directly imposes constraints into the discrete diffusion sampling process, resulting in a training-free and effective approach. Experiments in toxicity-controlled text generation, property-constrained molecule design, and instruction-constrained text completion demonstrate that CDD achieves zero constraint violations in a diverse array of tasks while preserving fluency, novelty, and coherence while outperforming autoregressive and existing discrete diffusion approaches.