Interpolating Discrete Diffusion Models with Controllable Resampling
For practitioners of generative modeling, IDDM offers a new approach to improve discrete diffusion, though improvements are incremental as results are competitive but not SOTA.
IDDM introduces a controllable resampling mechanism for discrete diffusion models that reduces dependence on intermediate latent states, mitigating error accumulation. It achieves competitive performance on molecular graph and text generation tasks against state-of-the-art models.
Discrete diffusion models form a powerful class of generative models across diverse domains, including text and graphs. However, existing approaches face fundamental limitations. Masked diffusion models suffer from irreversible errors due to early unmasking, while uniform diffusion models, despite enabling self-correction, often yield low-quality samples due to their strong reliance on intermediate latent states. We introduce IDDM, an Interpolating Discrete Diffusion Model, that improves diffusion by reducing dependence on intermediate latent states. Central to IDDM is a controllable resampling mechanism that partially resets probability mass to the marginal distribution, mitigating error accumulation and enabling more effective token corrections. IDDM specifies a generative process whose transitions interpolate between staying at the current state, resampling from a prior, and flipping toward the target state, while enforcing marginal consistency and fully decoupling training from inference. We benchmark our model against state-of-the-art discrete diffusion models across molecular graph generation as well as text generation tasks, demonstrating competitive performance.