Authentic Discrete Diffusion Model
This addresses a fundamental limitation in discrete diffusion models for researchers in generative AI, though it appears incremental relative to prior pseudo-discrete approaches.
The authors tackled the problem of pseudo-discrete diffusion methods by proposing an Authentic Discrete Diffusion (ADD) framework that preserves diffusion characteristics directly in one-hot space, achieving superior performance on classification tasks and excellent text generation on image captioning.
We propose an Authentic Discrete Diffusion (ADD) framework that fundamentally redefines prior pseudo-discrete approaches by preserving core diffusion characteristics directly in the one-hot space through a suite of coordinated mechanisms. Unlike conventional "pseudo" discrete diffusion (PDD) methods, ADD reformulates the diffusion input by directly using float-encoded one-hot class data, without relying on diffusing in the continuous latent spaces or masking policies. At its core, a timestep-conditioned cross-entropy loss is introduced between the diffusion model's outputs and the original one-hot labels. This synergistic design establishes a bridge between discriminative and generative learning. Our experiments demonstrate that ADD not only achieves superior performance on classification tasks compared to the baseline, but also exhibits excellent text generation capabilities on Image captioning. Extensive ablations validate the measurable gains of each component.